Europe will not be able to preserve its living standards if it has to absorb millions of migrants from Africa, the foreign secretary has said.
Philip Hammond called for EU laws to be overhauled to make sure that people coming from Africa to Europe could be returned to their home country.
Shadow foreign secretary Hilary Benn accused Mr Hammond of "scaremongering".
Steve Symonds of Amnesty International UK described Mr Hammond's "mean-spirited response" as "shameful".
"Rather than throwing up the drawbridge and talking about how Europe can 'protect' itself from migrants, Mr Hammond should be working with our EU partners to ensure that people don't drown in the Mediterranean or get crushed beneath lorries at Calais," he said.
Mr Benn added that Mr Hammond should "think more carefully about what he says".
"We need responsibility and international leadership from this government, not scaremongering," he said. "Those fleeing Syria are desperate refugees from a country being torn apart by war."
Speaking during a visit to Singapore, Mr Hammond said: "So long as the European Union's laws are the way they are, many of them will only have to set foot in Europe to be pretty confident that they will never be returned to their country of origin.
"Now, that is not a sustainable situation because Europe can't protect itself and preserve its standard of living and social infrastructure, if it has to absorb millions of migrants from Africa."
He said the problem had to ultimately be resolved by being able to return those who are not entitled to claim asylum back to their countries of origin - something Mr Hammond described as "our number one priority".
This, he added, was key to resolving the "crisis" at Calais, where thousands are gathered in the hope of crossing the Channel into the UK.
There have been increased security measures introduced in Calais in recent weeks, after hundreds of attempts by migrants to break into Channel Tunnel terminals.
On Saturday, 18 suspected illegal immigrants were arrested after a lorry was stopped on the M1 in Hertfordshire.
Mr Hammond went on to say more could be done to protect the Channel Tunnel from illegal migrants.
"Having reviewed the situation in the light of the crisis, it is clear that there is more that can be done to enhance the physical security of the tunnel.
"But we also have to work with our French colleagues to try to deal with the root cause of the problem. So long as there are large numbers of pretty desperate migrants marauding around the area there will always be a threat to the tunnel's security."
A new fence about a mile long, built to stop migrants from getting to the UK via the Channel Tunnel, has been built in Calais and was paid for out of £7m made available by the British government.