Kilmarnock shocked Rangers and breathed new life into the title race as Manuel Pascali's header gave them a first home win over the Ibrox side since 1994.
Ally McCoist's team were looking to equal an 82-year-old club record by remaining unbeaten on the road for a 15th consecutive game.
However, Kilmarnock had other ideas and Pascali's 80th-minute header from Garry Hay's lofted pass secured victory.
Rangers now lead Celtic by four points after their first loss of the season.
Within the first minute, debutant Sone Aluko supplied Steven Davis inside the box, but his cross from the right for Nikica Jelavic was just too strong and the Croatian could not connect in front of goal.
Aluko looked in the mood and played in Jelavic who hit the post, although the offside flag had gone up for a tight decision.
Kilmarnock, forced to substitute the injured midfielder Danny Buijs for Danny Racchi after seven minutes, briefly threatened from a rare foray upfield as Pascali saw a header blocked from a corner.
Kenny Shiels' side were soon in trouble again as goalkeeper Cammy Bell was challenged by Jelavic as he prepared to kick clear, but the ball bounced out for a throw-in.
The Ayrshire side slowly grew in confidence as the game remained goalless and Rangers were dealt a blow as the injured Kyle Lafferty had to be replaced by youngster Kane Hemmings.
The Rugby Park side's Gary Harkins went down after a shoulder-to-shoulder tussle with Kirk Broadfoot in the penalty box, but the referee was unmoved.
At the other end, McCoist's side had a shot from Hemmings which stung the palms of Bell.
Rangers then had a strong penalty claim on the brink of half-time when Hay took out Aluko in the box in a goalmouth scramble before McCulloch's effort hit the side netting.
Kilmarnock made a strong start to the second period and Dean Shiels' through ball found Harkins only for Steven Whittaker to block his effort and concede a corner.
From the resultant set-piece by Shiels, the ball cracked off Jelavic's head and smacked against his own team's crossbar with Allan McGregor beaten.
It was the Ibrox side's turn to threaten next as Hemmings released Aluko, whose powerful effort was cleared off the line by James Fowler with Bell beaten.
The action switched to the other end after 65 minutes when Liam Kelly's defence-splitting pass sent Paul Heffernan one-on-one with McGregor, but the Rangers No 1 stretched out his left hand to divert the ball clear.
McCoist's men had a fantastic chance to take the lead when handed a free kick 20 yards from goal, but Davis could only fire straight into the opposition wall.
Aluko still had plenty of energy left and his left-foot shot drive trundled into the box and hit the base of the post, but Hemmings could not convert the rebound.
The miss was to prove costly as Kilmarnock swept upfield and Hay lofted a hopeful ball into the box from around 25 yards.
Aluko failed to step out with the rest of Rangers' defence and Pascali connected to head past McGregor and put the home side ahead.
Substitute Alejandro Bedoya shot straight at Bell from inside the box in the closing seconds as Kilmarnock held on for a historic victory.