Relegation strugglers Bury stunned managerless Swindon and ended the Robins' 13-match unbeaten run.
Steven Schumacher scored the only goal of the game, turning in Joe Skarz's cross to secure the Shakers their third away win of the season.
Swindon looked lost without the influence of ex-boss Paolo Di Canio and created few chances.
James Collins volleyed wide in the second half, while Aden Flint's header was well saved by keeper Trevor Carson.
At the other end, Wes Foderingham was forced in to action after Schumacher's powerful shot.
The result means Swindon slide down to fourth in League One, while Bury remain 22nd.
VIEWS FROM THE DRESSING ROOM
Swindon Town caretaker manager Tommy Miller told BBC Wiltshire:
"We were rubbish throughout. We didn't get going at all, and I include myself in this.
"You wouldn't have thought the top of the league was at stake.
"Maybe it was down to complacency because Bury are struggling and it would have been an easy three points.
"We weren't at the races and it was very disappointing."
Bury manager Kevin Blackwell told BBC Radio Manchester:
"We knew we had to be well organised because they're good going forward and they've scored a lot of goals this season.
"The players stuck to the game plan and that's what frustrates me. On Friday night after 60 minutes, we looked as though we were comfortable and then we shoot ourselves in the foot with stupid decisions.
"I feel like I should be standing here with six points but what a way to return. To come to the team second in the table and do that to them was fantastic."