Ryan Catney levelled a minute from time to give the Reds a dramatic Premiership draw at Shamrock Park.
Ports striker Matthew Tipton scored twice from headers before Diarmuid O'Carroll pulled one back.
Neil McCafferty was sent-off for two yellows on 72 and Martin Donnelly netted from the resulting penalty.
Tipton completed his treble and Catney equalised with a lob before Reds keeper John Connelly was red-carded for handling outside the area.
Cliftonville forward Chris Scannell squandered three early chances before Tipton headed in on 12 minutes from an Andrew Burns cross.
It was a repeat finish two minutes later with Ross Redman the provider from a corner.
Reds midfielder Barry Johnston hit the inside of a post and while televsion replays showed that the ball then crossed the line, a goal was not awarded.
O'Carroll's superb volley into the top corner from a free-kick came after 53 minutes.
A foul on Catney inside the area earned McCafferty a second yellow and Donnelly duly converted the equalising spot-kick.
Tipton fired into the bottom corner from a headed knock-down to put the Ports back in front with 15 minutes left.
Catney chipped the ball over David Miskelly to secure a point for Cliftonville, who ended a thrilling encounter with 10 men after Connelly handled a yard outside the box.
Portadown are now nine points behind pacesetters Linfield, who beat Carrick Rangers 4-1.