Darren Boyce curled home a spectacular effort in the 72nd minute to secure victory for Ballymena at Dungannon.
Jamie Tomelty had given the Swifts the lead with a free-kick from the edge of the box on seven minutes but Jim Ervin levelled from six yards on 22 minutes.
Allan Jenkins finished off David Cushley's cross from the left after 38 but Gary Liggett nodded in from a Tomelty corner in the 53rd minute.
Sixth-placed Ballymena are now eight points ahead of Glenavon and Dungannon.
Liggett won the free-kick which saw Tomelty find the net for the first time since re-joining the Stangmore Park club with a low free-kick into the bottom right hand corner.
Defender Ervin's neat turn and finish was his first goal since joining Ballymena from Linfield earlier this month.
Jenkins continued his impressive form to put the visitors ahead at the interval, but Liggett equalised against his former club to give the Swifts hope of a point.
Cushley provided the assist for Boyce to score his second goal in two games for the Sky Blues, who now face leaders Linfield on Saturday.