Hereford ended in-form Kidderminster's run of seven straight league wins as they gained revenge for their Boxing Day defeat at Edgar Street.
Josh O'Keefe struck the game's only goal with a 13th-minute penalty into the bottom corner after Josh Gowling had fouled Bulls debutant Chris Sharp.
Jamille Matt, Harriers' match-winner six days earlier, was denied an equaliser when he struck the woodwork.
Bulls keeper James Bittner then saved Kyle Storer's 52nd-minute penalty.
Bittner had done well to thwart Matt in first-half injury time before Andrew Corbett fouled Mickey Demetriou seven minutes after the restart.
But he then made another top-class stop from Storer's spot-kick to secure the points for Hereford, who climb a place to 12th in the table thanks to their first win in almost a month.
VIEW FROM THE DRESSING ROOM
Harriers manager Steve Burr told BBC Hereford and Worcester:
"I thought we played very well today and did everything but score. We're disappointed with the result but I was very pleased with the performance.
"I couldn't have asked for anything more from the players and I felt their goal did lead a charmed life.
"Our penalty came at a crucial time. That's three we have missed now and each one has cost us, so that's something we need to address."
Bulls manager Martin Foyle told BBC Hereford and Worcester:
"You can say it was a hard-earned win and it certainly was and you have got to pay compliments to Kidderminster.
"They were one of my tips at the start of the season and they're an outstanding side. They started slowly but they've got some momentum and had picked up seven straight wins in the Conference.
"We knew we were going to be in for a hard game today and that is exactly what it turned out to be."