Stephen Lowry drilled the ball through a crowded penalty area to score an 88th-minute winner for Coleraine.
Kevin Braniff opened the scoring with a low diving header but Johnny Black levelled with a stunning left-foot free-kick from inside his own half.
Tim Mouncey headed in but Shane Jennings found the net with a fabulous low finish before half-time.
A Jennings header made it 3-2 and Jamie Tomelty equalised with a snapshot while Shane McCabe was sent-off late on.
McCabe received a straight red card for a clumsy challenge on Leon Knight, followed by dissent.
The first half of this seven-goal thriller was a pulsating affair and Braniff got the goalscoring off to an early start by meeting Matthew Tipton's cross in the fifth minute.
Black's sweeping long-range free-kick over the head of Miskelly brought the Bannsiders level on 17 minutes but Mouncey headed in Tomelty's cross from six yards out three minutes before the break.
Jennings caught Sean Mackle in possession and dispatched a fabulous finish for his first goal for the club on the stroke of half-time.
Jennings grabbed his second with a well-guided header from Black's looping free-kick after 65 minutes.
Former Linfield midfielder Tomelty thought he had earned his side a point but Lowry scored the late winner to secure his side's first league victory of the season.
Portadown drop to third place in the table, three points behind leaders Linfield and one point behind Cliftonville.