Freddie Burns missed a crucial 65th minute conversion for Gloucester as bottom side Newcastle earned only their second Premiership win of the season.
Olly Morgan went over in the first two minutes for Gloucester but the hosts hit back through Jordi Pasqualin's try.
James Simpson-Daniel and James Fitzpatrick then exchanged tries as the visitors led by three at half-time.
But Falcons went ahead through three Jimmy Gopperth penalties and Burns could not convert Scott Lawson's try.
The visitors had made a fantastic start after Gopperth, who was to score 16 points, sliced his touch kick into the strong wind.
Gloucester ran it back and Morgan cut inside to go over in the right corner, with Burns landing a difficult conversion.
Newcastle had an immediate chance to cut the deficit and Gopperth obliged with a well-taken penalty before the hosts took the lead in the seventh minute.
Mike Tindall lost possession for Gloucester and Michael Mayhew created a chance for youngster Pasqualin to score Newcastle's first try of the match against his former club, with Gopperth adding the extras.
But two successive penalties from Burns turned a three-point deficit into a three-point lead and that advantage was increased when the hosts were caught napping.
Simpson-Daniel broke through from left to right to score and Burns kicked his second conversion to make it 20-10 to the visitors.
They then had flanker Alasdair Strokosch sin-binned for a trip on Gopperth and Newcastle made it count.
Six minutes before the break Fitzpatrick went over and Gopperth's conversion meant Gloucester's lead was cut to three at half-time.
The hosts were level a minute into the second half as Gopperth kicked his second penalty of the game, and his third put Newcastle ahead for the first time since the seventh minute.
That lead was increased to six points as Gopperth was successful again just after the midway point of the second half.
A fantastic rolling maul by the visitors enabled hooker Lawson to go over and close the gap to one but, for the first time in the game, Burns was unsuccessful with the boot.
Gopperth had a chance to make the game safe with two minutes to go but he too ruined his 100% kicking record on the night as his penalty drifted to the right, but Newcastle held on.
Newcastle Falcons head coach Alan Tait:
"The yellow card was something of a turning point but I'm not surprised. We had a review of the refereeing after the Harlequins game and it was quite clear that Jimmy is a target and needs to be protected.
"We took 10 points while Strokosch was in the sin-bin and that proves how important yellow cards are in the game.
"It was not the best of starts by us and I thought 'here we go again' when we gifted them two tries, but you have to give the players credit for the fightback and they really grafted for the win."
Gloucester head coach Bryan Redpath told BBC Gloucestershire:
"We had a good start but then I was disappointed because our basics weren't good enough and that was probably the clear thing in the conditions. You needed good basics and some of them weren't.
"We conceded 10 points in the time Strokosch was in the bin. They kicked a penalty and scored a try off a line-out. We've talked about it - away from home you can't have yellow cards.
"You have to be very disciplined and we weren't. We gave one away straight after half-time for holding on and another for not releasing the tackler. They were the first six points of the second half."
Newcastle: Goosen; Hufanga, Uys, Fitzpatrick, Shortland; Gopperth, Pasqualin; Shiells, Mayhew, Murray; Hudson (capt), Van der Heijden; Tuifua, Welch, Hogg.
Replacements: Helleur for Uys (55), Pilgrim for Pasqualin (60), Wells for Shiells (63), Graham for M. Mayhew (60), Fondse for van der Heijden (55), Wilson for Tu'ifua (65). Not Used: Frazier, Catterick.
Gloucester: Morgan; Simpson-Daniel, Tindall, Fuimaono-Sapolu, Voyce; Burns, Lewis; Wood, Lawson, Harden; James, Hamilton; Strokosch, Qera, Narraway (capt).
Replacements: Trinder for Tindall (70). Not Used: Cortese, Murphy, Chistolini, Deacon, R. Lawson, Taylor, Knight.
Sin Bin: Strokosch (33).
Referee: Greg Garner (RFU).