Southport fought back from two goals down to leave Hereford winless in six Blue Square Bet Premier games.
Samuel Clucas put the Bulls in front from close range and they doubled their lead with a Ryan Bowman penalty after Simon Grand fouled James McQuilkin.
Karl Ledsham scored direct from a corner to raise Southport's hopes.
And the comeback was complete when Chris Lever scored from the spot following Chris Todd's challenge on Steven Tames.
VIEWS FROM THE DRESSING ROOM
Hereford manager Martin Foyle told BBC Hereford and Worcester:
"It was outstanding - I thought we played some good stuff and the formation worked as we were doing all the right things that we had been telling the players to do.
"We told them to just have the confidence to cuddle balls into the corners as we have got willing runners in Sam Clucas, Ryan Bowman and Phil Marsh.
"But then after that we stopped doing the good things - we started taking extra touches in midfield and then were getting caught and robbed.
"All of a sudden it was a free kick and then it was in our box and suddenly it was chance after chance for them."