Abdulai Bell-Baggie grabbed a late winner in extra time to send Tranmere in to the League Cup second round.
All the drama came late on after neither side could break the deadlock in normal time.
Cole Stockton gave the visitors the lead in the 93rd minute when he lashed in to the top corner from 12 yards out.
Chesterfield veteran Jack Lester levelled the score with a chipped effort but substitute Bell-Baggie secured the win at the death.
VIEW FROM THE DRESSING ROOM
Tranmere Rovers manager Ronnie Moore told BBC Radio Merseyside:
"I think we deserved it and the disappointing thing for me was that we had to go into extra time.
"Give our lads their due, we've kept going and you won't see a better goal that Bell-Baggie's.
"We had six players missing, including three centre-halves.
"For Ben Gibson to travel from Middlesbrough today, meet everyone in there, put on a shirt and then come out and play was unbelievable."