Toast Of New York had to settle for a sixth-placed finish in the inaugural running of the $1.25m Belmont Derby Invitational Stakes.
The Michael Buckley-owned colt skipped Epsom to enter the race, staged at Belmont Park, New York.
Ridden by Jamie Spencer, Toast Of New York threatened briefly in the straight but Shug McGaughey's Mr Speaker ran out a surprise winner.
Aidan O'Brien's Adelaide was a close second, with Flamboyant third.
Earlier in the week, Toast of New York's trainer Jamie Osborne defended his plans to run his star horse - which won the $2m UAE Derby in Dubai in March - on the diuretic Lasix.
The medication, used to reduce the chance of blood vessels bursting, is barred on race-days in Britain but regularly used on horses in the US.