Swindon twice let the lead slip to allow struggling Hereford a share of the spoils.
Goals from Raffaele De Vita and Mehdi Kerrouche put the hosts in charge before Tom Barkhuizen smashed the ball home from the edge of the box.
Substitute Yoann Arquin headed home unmarked from six yards for the equaliser but Simon Ferry finished off a neat move to restore Swindon's lead.
However, Harry Pell came off the bench to nod home in the dying seconds.
VIEWS FROM THE DRESSING ROOM
Swindon manager Paolo di Canio told BBC Wiltshire:
"In the first half I told my players I don't like this atmosphere.
"We were 2-0 up, but we had to be capable of scoring another goal.
"At the end, one point is good because we didn't deserve to win."
Hereford United midfielder Harry Pell told BBC Hereford and Worcester:
"We came in at 2-0 down and we didn't think that we had done too badly but that has been the case all season.
"It is just belief, really, today there was so much self-belief and we looked a different team.
"We just need to continue it and hopefully we will do so throughout the season now."