Sone Aluko's stunning volley gave Hull victory over Newcastle in a highly entertaining encounter.
Loic Remy gave the hosts the lead when he headed home from Papiss Cisse's miscued shot at the back post.
Robbie Brady levelled with a low effort before Remy scored his second when Yohan Cabaye's shot fell into his path.
Brady's free-kick was glanced in by Ahmed Elmohamady shortly after the restart and Aluko won it with a superb finish from 18 yards.
After a slow start to the season - the Magpies did not manage a shot on target in their first two outings - Alan Pardew's side came into this match on the back of successive league wins over Fulham and Aston Villa.
Hull have made a steady start to their first season back in the top flight after three years in the Championship, having picked up four points from their opening four games.
The hosts began the game in confident mood and went ahead in the 10th minute when Moussa Sissokho's cross to the back post was hit into the ground by Cisse. His attempted shot turned into the perfect ball for Remy to head home.
The goal was deserved reward for Pardew's side, who made a bright start to the match as they looked to secure a fourth successive win in all competitions.
However, they started to allow errors into their game and, after goalkeeper Tim Krul did brilliantly to deny Danny Graham from a Brady cross, they conceded their first goal at St James' Park this season.
Newcastle failed to clear following the Dutchman's stop and, with the defence out of position, Aluko threaded a neat pass to the excellent Brady, who struck a low shot through the legs of Krul for his third goal of the season.
Steve Bruce's side had chances to take the lead but Graham, looking for a first league goal since New Year's Day, failed to control a Tom Huddlestone pass when in on goal before Jake Livermore saw his shot well blocked by Matthieu Debuchy.
Newcastle went back in front against the run of play just before the break in fortuitous circumstances. Cabaye's shot from 25 yards hit Hull defender James Chester and Remy confidently slotted home his second.
The goal was harsh on the visitors but they responded well and levelled once more.
Aluko was fouled by Vurnon Anita and, from the resulting free-kick, Brady found the unmarked Elmohamady, who saw his header go in off the post.
With Remy tiring and Cabaye forced off after a heavy challenge from Curtis Davies the hosts began to run out of ideas and they fell behind to Aluko's brilliance.
From George Boyd's cross, Nigerian Aluko superbly guiding his effort in off the same post as Elmohamady's header.
As the game entered five minutes of injury-time Remy had the chance to score his third but he fired wide from six yards out and Hull held out for a first away win of the season.
Newcastle boss Alan Pardew:
"We started really, really well but then we dropped down a gear after our goal and that caused us problems for the rest of the game. Third gear is not enough in this league against any side.
"You look for consistency and this team is a good team but we lack a bit of consistency and that is what gets you results.
"I have to pay tribute to Hull, they did a lot of things very well today. We had a chance to pinch a point but it just fell to Loic Remy a bit quick.
"We have to regroup and make sure we're ready for the next one."
Hull City boss Steve Bruce:
"I'm absolutely delighted I have to say. I've been really happy with the way we've played all season.
"We created a lot of chances today and we thoroughly deserved our win.
"I'm a Geordie at the end of the day and I've always enjoyed coming back, apart from last time (when his Sunderland side lost 5-1 to the Magpies) but today was very satisfying.
"George Boyd did wonderfully well and Aluko's shot went in off the post. I think he can light up this league."