Championship club Bangor hold the advantage over Warrenpoint after the Premiership play-off first leg.
Town striker Daniel Hughes hit the woodwork early on but Bangor made the breakthrough when Stephen Moan turned Andy Morrow's cross into his own net.
Town keeper Jonathan Parr denied Ethan Boylan before the break and both sides went close before Paul McDowell slotted in to make it 2-0 in added time.
The teams will meet at Milltown in Friday night's second leg.
Warrenpoint were first to threaten at the Bangor Fuels Arena with John McGuigan firing wide after keeper Darren Gibbons failed to control a Mark Hughes free-kick.
Town striker Daniel Hughes struck a post 12 minutes into the game and it a Warrenpoint player who netted the opener after 29 minutes.
Andy Hall was involved in the build-up which led to the unlucky Moan diverting Morrow's cross past Parr.
Boylan was sent clear from Andy Hall's pass but Parr did well to save and smother the ball.
Seasiders defender Dean Youle twice threatened early in the second half with a headed effort and half-volley.
Gibbons tipped over a Mark Hughes strike and Town had penalty claims turned down from the resulting corner after Colin Nixon's challenge on Conor McDonald.
Hall squandered a good opening for the hosts before substitute McDowell shot wide for the second time since his introduction.
But he made no mistake a minute into injury time, finding the bottom corner from Mark Cooling's pass.