Oldham midfielder Jose Baxter scored a late penalty to complete a second-half double to defeat Stevenage.
Stevenage went ahead when Darius Charles pounced on a loose ball.
James Tarkowski equalised for Oldham by crashing home a sweet volley before Baxter scored on the rebound after his initial spot-kick was saved.
Christian Montano put Oldham 3-1 up but Boro fought back and were level thanks to efforts from Greg Tansey and Filipe Morais before Baxter's late penalty.
VIEW FROM THE DRESSING ROOM
Stevenage boss Graham Westley told BBC Three Counties Radio:
"I went into the game a little bit apprehensive to be honest, knowing I had so many lads stepping up a level.
"I think we'll take the reaction to going 3-1 down, the quality of what they did at that stage is a good sign of the character that's there
"The mistakes that went on during the 90 are pretty easy to eradicate and when we've had games together I don't think we'll be seeing the type of errors that we saw today.
"You couldn't say there was any malice in any of the challenges but there were far too many cards."