Four goals were scored in stoppage time as Alfreton ended their 13-game winless run by beating Hayes and Yeading.
Anthony Church hit the bar and missed a penalty for Town, who had Jamie Mullan sent off for lashing out, before Nathan Arnold's drilled finish for them.
Arnold scored a 92nd-minute own goal before Jake Moult's curler for Town.
There was still time for Louie Soares to curl home a second Hayes equaliser and then for Nathan Jarman to head a 96h-minute winner for Alfreton.
"The players were magnificent," said Alfreton manager Nicky Law.
Alfreton remain next-to-bottom in the Blue Square Bet Premier table, but Law believes his side's dramatic win can act as a springboard for the rest of the season.
"That we won was down to the determination of the side," he added on the club's official website.
"We took the game to them, the players were magnificent and never gave up.
"Our fitness levels were excellent again and carried us through to a win we had to get."