Danny Hylton's 89th-minute winner sent 10-man West Ham crashing out of the Carling Cup and put League Two side Aldershot into the second round.
The Hammers led through a curling shot from Junior Stanislas but had debutant Callum McNaughton sent off for a professional foul after the break.
That changed the game and Luke Guttridge equalised from close range after Ruud Boffin denied Alex Rodman.
Hylton snatched victory from 12 yards out after a goalmouth scramble.
That gave Aldershot their first win in this competition since the club reformed in 1992 and returned to the Football League in 2008.
But for the third time this season West Ham were left to rue conceding an important late goal at Upton Park.
Hammers boss Sam Allardyce made nine changes from the team that were held at home by Leeds in the Championship on Sunday and initially his side struggled to settle.
It took took a superb save from Boffin to deny Jake Taylor when he ran through but West Ham weathered the storm and took the lead with a strike of real quality from Stanislas.
The Shots were given hope when McNaughton scythed down Hylton after 47 minutes.
And Dean Holdsworth's side piled on the pressure before Guttridge pounced to level with 12 minutes to go, reacting quickly after Boffin blocked Rodman's shot.
With West Ham hanging on, Michael Rankine and Jermaine McGlashan went close to winning it for Aldershot before Hylton found the bottom corner with a low angled shot.
Matt Taylor came close to equalising in stoppage time but could only fire into the side netting after latching on to a flick.