Bury are still searching for their first win of the season following defeat at home by Notts County.
The Shakers were behind after just two minutes when Lee Hughes diverted Alan Judge's free-kick into the net.
Lateef Elford-Alliyu then had a great chance for the hosts but headed straight at keeper Bartosz Białkowski.
And with Bury pressing for an equaliser, a quick Magpies break allowed Judge to finish past Trevor Carson and seal victory.
VIEWS FROM THE DRESSING ROOM
Notts County manager Keith Curle told BBC Radio Nottingham:
"Credit to Bury, they stuck to their game-plan, flooding the midfield, good movement, good rotation in their play.
"Even though we got the early start from a free-kick that went in, they caused us problems.
"There's a unity building and you get it from coming to places like Bury.
"Make no mistake, this is not an easy place to come to."