A penalty from Diarmuid O'Carroll three minutes into injury-time secured a point for Crusaders and maintained their seven-point lead at the top.
Mark Hughes scored for the hosts in the 72nd minute with a low right-foot shot from the edge of the box.
Josh Robinson was sent-off on 87 but O'Carroll levelled after Paul Heatley was brought down by Liam Bagnall.
Ryan Harpur scored a hat-trick as Dungannon leapfrogged Ballymena into eighth with a 4-2 win at Warden Street.
For the Sky Blues, it was a fifth consecutive league defeat, while the Swifts have now taken 10 points from their last four Premiership outings.
Andy Mitchell swivelled and volleyed into the net when the ball fell to him in the fifth minute, for his 13th goal of the season.
Harpur netted a 15th-minute penalty after he was fouled by Dwayne Nelson, then David Cushley pulled one back with a low left-foot shot into the bottom corner after good build-up play.
Allan Jenkins equalised by rounding Andy Coleman and slotting into the empty net a minute before the interval.
Harpur dinked the ball over Nelson as he came off his line in the 54th minute and completed his treble three minutes later by slotting in left-footed.
Warrenpoint in good form
Warrenpoint went into their game against the champions at Milltown with a record of five wins and three draws from their previous nine league fixtures, having conceded just once in six Premiership matches.
In the first half, Heatley headed over from a Forsythe cross and the latter then nodded over from Michael Carvill's delivery.
Hughes's strike looked to have earned Barry Gray's side victory and Crusaders' woes deepened when Robinson was dismissed for bringing down John McGuigan as he bore down on goal.
Substitute O'Carroll's late spot-kick salvaged an unlikely point for Stephen Baxter's outfit and 'Point remain 11th in the table.
|Danske Bank Irish Premiership|
|Ballymena United||2-4||Dungannon Swifts|