Bury recorded their first win of the season to climb off the bottom of League One and put Hartlepool at the foot of the table.
Both sides created numerous first-half chances but Tom Hopper opened for Bury, converting a low cross from on-loan Leeds teenager Dominic Poleon.
David Worrall doubled the advantage with a superb strike.
Ryan Noble pulled one back with a 30-yard shot only for Neil Austin to waste a late penalty and with it a point.
The full-back struck his spot-kick against the crossbar after Noble was fouled in the area.
VIEW FROM THE DRESSING ROOM
Bury manager Kevin Blackwell told BBC Radio Manchester:
"I've not been here for the 14 games, but I know that we've been very close [to a win] in the four or five games that I've been here.
"I knew we'd win a game sooner rather than later, but it's nice to win.
"I'm delighted for the boys because they're off the bottom of the table.
"They've kept going and you've got to give them credit. It's easy to say 'this isn't happening' and throw the towel in. They didn't do that and they had to hang in tonight."
Hartlepool United first-team coach Neale Cooper tells BBC Tees:
"The damage was done by then, it was a well taken goal by Ryan Noble that gets us back to 2-1 but it's not good enough.
"It's a hard place I'm in at the moment, we'll have the journey home tonight and I'll think about it in the morning.
"I've always had a good affection with the Hartlepool people but now, and they've every right to, I'm getting a hard time and that's not pleasant. I've said to the boys, we've had a long chat in the changing room that they've got to get better."