Marc Sneyd scored 11 points against Hull FC to help keep Castleford's perfect start to the season intact.
Gareth Ellis put the visitors 12-6 up at the break after Fetuli Talanoa opened the scoring early and Michael Shenton went over for Tigers.
Sneyd then converted his own try to level, only for Liam Colbon to restore Hull's lead with a try on his debut.
Frankie Mariano equalised late on, with Sneyd adding the extras and a drop goal to seal victory.
Castleford's triumph keeps the pressure on Super League leaders St Helens, who are the only other side with five wins from five matches to start the season.
Defeat is Hulls FC's third this term and comes after a thumping 44-16 victory against Bradford last week, and a two-point loss to Warrington earlier this month.
Kirk Yeaman's quick pass on the last tackle was enough to breach the Tigers' defence for the first time, to put Talanoa over inside three minutes.
Castleford had to wait until the fifth minute just to get their hands on the ball.
Daryl Clark made the wait worthwhile, breaking through the Black and Whites' line before Grant Millington sent Shenton racing clear.
Ellis punished Jordan Tansey for a mistake in his own in-goal area, eventually crashing over from close range, and Ben Crooks made it two from two with the boot to restore their six-point advantage.
The visitors were dealt a blow before the break as prop Josh Bowden suffered an ankle injury.
Mariano laid the platform for Sneyd to dummy and hit back for Castleford with the first try of the second half.
Colbon finished brilliantly in the corner, following good work from Richard Whiting and former Tiger Joe Westerman, to edge Hull FC in front once again on 56 minutes. Crooks missed the decisive conversion, and was left in a heap soon after following a big hit by Justin Carney.
Shenton then burst down the middle and found Mariano in support three minutes from time to restore parity before Sneyd teed up his third conversion and further stamped his authority on the match with a drop goal.
Castleford head coach Daryl Powell:
"I don't think we were great today, to be honest. It was probably our worst performance of the season.
"I thought Hull played well and put us under pressure. But we found a way to win and the character of the side is obviously fantastic.
"We'll learn a lot of lessons from it as well, which I think is great for us.
"Hull set up to defend us in a certain way and we didn't handle it real well, so if it happens to us again we'll hopefully learn the lesson.
"He's coming on, Sneydy [Marc Sneyd]. I thought his passing game wasn't fantastic in the first half, but I think we only had two sets on their line.
"He's a pretty cool, calm sort of character."
Hull FC coach Lee Radford:
"They are the cruel ones to take. I thought some of the efforts from some of the players were fantastic and we probably deserved more out of the game than what we got.
"They [Castleford] go on bouncing and obviously we've lost another close one which is disappointing.
"It's becoming a bit of a habit, that was probably the third time it's happened, so it's something we want to eradicate real quick."
Castleford: Tansey; Owen, Channing, Shenton, Carney; Sneyd, Finn; Lynch, Clark, Huby, Holmes, Hauraki, Millington.
Replacements: Webster, Milner, Massey, Mariano.
Hull FC: Shaul; Colbon, Crooks, Yeaman, Talanoa; Heremaia, Horne; Paea, Houghton, Bowden, Ellis, Tuson, Westerman.
Replacements: Watts, Whiting, Thompson, Rankin.
Referee: Tim Roby (RFL)