Alan Connell scored twice as Swindon claimed their first win in four and ended Rotherham's unbeaten run.
Alex Revell put the visitors ahead with a powerful header but Swindon hit back through Matt Ritchie's strike.
Revell nodded the Millers in front but they were reduced to 10 men as Danny Schofield received his second yellow.
Substitute Connell headed an equaliser and nodded in the winner for Swindon, before team-mate Alberto Comazzi was also dismissed for a second caution.
After four straight defeats in the league, and off-field problems involving manager Di Canio and now-departing striker Leon Clarke, Swindon were desperate to focus on sealing a victory, but faced an uphill battle against Rotherham, who had dropped just two points out of their five games and netted an impressive 12 goals.
It was former Swindon loanee Revell, who joined the Millers on transfer deadline day from Leyton Orient, who broke the deadlock with a glancing header from captain Ryan Cresswell's shot.
The Robins kept pushing for an equaliser and it paid off when Ritchie found the end of Mehdi Kerrouche's cross and fired home a shot which was helped in off the body of Tom Newey.
And winger Ritchie continued to cause the Rotherham defence problems in the second half but missed the chance to put his side in front for the first time, scuffing his shot from five yards out after Raffaele De Vita had skipped up the wing and provided a delicate cross.
But once again the Millers went ahead when De Vita conceded a free-kick on the edge of the area which was whipped across by Schofield and headed in by Revell for his second of the game.
After the Millers lost Schofield to his second booking, Connell came off the bench and within seconds he had brought Swindon level again with a header at the far post, his first goal for the club.
And the out-of-favour striker proved his worth with his second from Ritchie's pin-point cross, to leave Andy Scott's side stunned.
Comazzi's dismissal for his second caution, followed by a dug-out scuffle between a member of Di Canio's coaching staff and Rotherham full-back Dale Tonge, could not dampen the spirits at a jubilant County Ground.
VIEWS FROM THE DRESSING ROOM
Swindon Town manager Paolo di Canio told BBC Wiltshire:
"We deserved to win. We tried to play the ball on the ground.
The lads were amazing.
Now we need to win some more games and then maybe we can play with more confidence."
Rotherham boss Andy Scott told BBC Radio Sheffield:
"I didn't enjoy it at all. From start to finish, we didn't play well.
"We made some poor decisions defensively and we kept on playing backwards.
"We got nothing out of it, but we'll pick ourselves up."