Celtic completed an aggregate victory against HJK Helsinki to progress to the Champions League play-off round.
Scott Brown threatened as the visitors sought to cancel out HJK's away goal and the hosts' Sebastian Sorsa and Mathias Lindstrom had efforts saved.
Ledley chested the ball past Ville Wallen to give Celtic the lead on the night in the 68th minute.
A second booking meant a dismissal for Victor Wanyama but the 10 men added to their lead with Georgios Samaras' shot.
Neil Lennon's side will discover who they will face in the next round when the draw is made on Friday.
Celtic captain Brown, a pre-match fitness doubt, beat the offside trap to latch on to Charlie Mulgrew's terrific lofted pass but goalkeeper Wallen was untested as the midfielder placed his shot wide.
The artificial surface seemed to be posing no problems for the Scottish champions and Samaras chested down another high ball before flashing a dipping volley past Wallen's left-hand post.
The Finns were struggling to carve out chances and Demba Savage failed to connect with Sorsa's cross in their best first-half attack.
But HJK pressed hard at the beginning of the second period and Sorsa's low drive from the edge of the box was turned out by Forster.
Kris Commons' curling free-kick was met by Wanyama and Celtic's claims of handball after the header had been blocked were turned down by Austrian referee Robert Schorgenhofer.
Lindstrom was unchallenged at Rasmus Schuller's corner and the defender nodded the ball into the ground only for Forster to awkwardly usher it out on the bounce.
The breakthrough arrived when Mulgrew found himself forward on the right in a Celtic break and his inswinging cross was forced past Wallen by the chest of Ledley.
But Celtic soon found themselves down to 10 men as Wanyama's foul on Mika Vayrynen resulted in a second yellow card and a suspension for the Kenyan.
It did not upset Celtic's rhythm though and Ledley's perseverance on the left was rewarded when Samaras collected the Welshman's cutback and placed a low shot into the net to confirm the Glasgow side's first away European win in three years.
Progress ensures Celtic will play group stage football this season with a parachute into the Europa League available if they lose their Champions League play-off.