An injury-time goal from striker Georgios Samaras gave Celtic a historic first away win in the group stages of the Champions League.
Gary Hooper put the visitors in front on 12 minutes before tap-ins by Emmanuel Emenike either side of half-time put Spartak Moscow 2-1 up.
After Juan Insaurralde was sent off for a professional foul on Hooper, Dmitri Kombarov's own goal drew Celtic level.
Samaras headed home Emilio Izaguirre's cross in the 90th minute.
Neil Lennon's side now lie second in Group G on four points, with the Russians pointless after two matches.
The win, their first group stage victory on the road in 19 attempts, sets Celtic up for a double header against Spanish giants Barcelona, starting at Camp Nou on 23 October.
Spartak set their stall out early on and tried to catch their visitors off guard with a series of crosses and corners.
But Celtic endured the pressure and took the lead with their first attack of the game, started with Victor Wanyama's forceful play in midfield.
Mikael Lustig was released on the right and Hooper got in front of Nicolas Pareja to convert the Swede's low cross.
Lennon's side soon found themselves defending in numbers again, one drive from former Celtic winger Aiden McGeady diverted out for a corner.
But Spartak were vulnerable to counter attacks, with wide men Kris Commons and the fit-again Samaras able to make ground on the artificial surface.
Hooper had the ball in the net again with a header from a Samaras cross but the goal was ruled out for offside.
A free-kick around 25 yards out gave Spartak a chance to level but Pareja's curling effort was kept out by a diving Fraser Forster.
However, the home side's next attack produced the equaliser, Emenike scoring from close range after Ari cut the ball back from Kim Kallstrom's perfectly weighted ball in behind Kelvin Wilson.
Another close-range finish by Emenike edged the hosts in front just after the break.
Forster should perhaps have done better than parry Ari's low strike into the path of Demy de Zeuww, who turned the ball back across goal for Emenike to touch home.
Spartak went down to 10 men when Insaurralde's pull on Hooper eventually resulted in a red card for the Argentine defender, French referee Tony Chapron overturning his own decision to let play continue after taking advice from one of his assistants.
After Spartak had taken off midfielder de Zeuww and sent on Sergei Bryzgalov to shore up the defence, Charlie Mulgrew's effort from the resulting free-kick was punched away by Sergei Pesiakov.
Lennon also made a change, with James Forrest replacing Wanyama and playing a crucial role in Celtic's second.
Samaras dummied Mulgrew's cross for Hooper, who set up Forrest on the right-hand side of the box.
The winger's shot was blocked by a foot of keeper Sergei Pesiakov but ricocheted into the net off Kombarov.
Celtic continued to press and snatched a deserved victory when Samaras looped a header past Pesiakov.