A statue of noted slaveholder Robert Milligan has been removed from outside the Museum of London Docklands.
Sadiq Khan earlier announced a review of all of London's statues and street names, saying any with links to slavery "should be taken down".
On Sunday, anti-racism protesters in Bristol tore down a statue of slave trader Edward Colston.
Milligan's monument was removed to "recognise the wishes of the community" said the Canal and River Trust.
There were cheers and clapping as the monument was lifted from its plinth using a crane.
The Museum of London Docklands said the statue of the prominent British slave trader, who owned two sugar plantations and 526 slaves in Jamaica, had "stood uncomfortably" outside its premises "for a long time".
"The Museum of London recognises that the monument is part of the ongoing problematic regime of white-washing history, which disregards the pain of those who are still wrestling with the remnants of the crimes Milligan committed against humanity," they added.
The Canal and River Trust said it had worked with the London Borough of Tower Hamlets, the museum and partners in Canary Wharf to have it removed.
As the Milligan statue was lowered from its plinth, thousands of people gathered outside an Oxford college to demand the removal of a statue of imperialist Cecil Rhodes.
Mr Khan said London had to face "an uncomfortable truth" with its historical links to slavery.
The Commission for Diversity in the Public Realm will review the city's landmarks - including murals, street art, street names, statues and other memorials - and consider which legacies should be celebrated before making recommendations.
Mr Khan said London was "one of the most diverse cities in the world", but said recent Black Lives Matter protests had highlighted that the city's statues, plaques and street names largely reflect Victorian Britain.
"It is an uncomfortable truth that our nation and city owes a large part of its wealth to its role in the slave trade," he said.
"While this is reflected in our public realm, the contribution of many of our communities to life in our capital has been wilfully ignored."
During a Black Lives Matter protest in central London on Sunday, a statue of Sir Winston Churchill in Parliament Square was sprayed with graffiti.
But Mr Khan said he did not consider statues of the likes of Churchill to be included in the review.
He said pupils needed to be educated about famous figures "warts and all" and that "nobody was perfect", including the likes of Churchill, Gandhi and Malcolm X.
Mr Khan told BBC Radio 4's Today programme he did not have ownership of the statues and the land they are on.
He also said it would be "inappropriate" to single out which statues and street names he thinks should go.
Instead a number of new memorials in the capital have been pledged by Mr Khan, including ones for Stephen Lawrence, the Windrush generation, a National Slavery Museum or memorial and a National Sikh War Memorial.
The Local Government Association's (LGA) Labour group has also announced that Labour councils across England and Wales are to review "the appropriateness" of monuments and statues in their towns and cities.
Campaigns calling for the removal or amendment of monuments celebrating controversial figures have increased in volume around the UK in recent days.
In Oxford, 26 councillors and an MP have called for a statue of imperialist Cecil Rhodes to be removed from an Oxford University college.
A plaque is to be added to a Henry Dundas monument in Edinburgh to "reflect" the city's links with slavery, while the leader of Cardiff Council said he would support the removal of a statue of slave-owner Sir Thomas Picton from the city's civic building.
London statues with links to slavery
- Guy made his money from the the stock market bubble of 1720. He had shares in the South Sea company when it started trading in 1711, with slaves sold to the Spanish colonies.
- Robert Milligan was a noted West Indian merchant, slaveholder and founder of London's global trade hub, West India Docks.
- Aside from founding an educational charity, Sir John Cass was a major figure in the Royal African Company, which was involved in the Atlantic slave economy.