Ballymena hit Dungannon's hopes of avoiding the drop in game which saw Swifts player/boss Darren Murphy intentionally miss an early penalty.
Gavin Taggart caught the ball after the players stopped as Chris Rodgers lay injured, but a spot-kick was given and a sporting Murphy passed to the keeper.
Alan Teggart, Gary McCutcheon and James Costello struck for United after the break but Craig McClean made it 3-1.
Johnny Montgomery set up a tense finish before Jamie Davidson sealed the win.
The penalty was awarded by referee Arnold Hunter after 15 minutes while Rodgers was stretchered off with a shoulder injury.
Sean Friars and Terry Fitzpatrick went close for the hosts in the first half before Teggart slotted home from Costello's pass after 50 minutes.
McCutcheon's glancing header hit the net and Costello added the third with a side-foot finish in the 64th minute.
McClean gave Dungannon hope with the goal of the game, a thunderous 25-yard strike into the top corner.
Montgomery rifled in after a goalmouth scramble six minutes from time to make it 3-2.
Swifts keeper Niall Morgan pushed up for a corner in the dying seconds and when the ball was cleared he was stranded as substitute Jamie Davidson netted from 20 yards.
Dungannon slip to second bottom and three points clear of Carrick Rangers, while United move to the top of Section B.