Defence Secretary Michael Fallon has said he "misspoke" when warning that some areas of the UK could be "swamped" by immigrants without changes to European Union rules on movement.
He told BBC Radio 5 live he had "used words I wouldn't normally have used".
His comments came after Germany appeared to rule out David Cameron's plan to limit EU arrivals in the UK.
Labour called the remark "desperate", while the Liberal Democrats said they were "not based on the facts".
Speaking in the House of Commons, Prime Minister David Cameron said Mr Fallon had been "absolutely right" to correct himself.
"It is right for politicians to raise concerns about immigration but we should always choose our language carefully," he told MPs. "He said this morning he wished he had chosen his language in a different way and I agree with that."
The prime minister made the comments in an update to Parliament about last week's EU leaders' summit, during which the UK was told it must pay an extra £1.7bn to the 28-member group. Mr Cameron has refused to do so.
The European Commission has argued the contribution revisions were calculated by independent statisticians using a standard formula agreed by all member states.
Mr Cameron's Conservative Party is facing a difficult by-election in Rochester and Strood, Kent, called after one of its MPs, Mark Reckless, defected to the anti-EU UK Independence Party.
Mr Cameron faced tough questions from Conservative MPs over immigration when he spoke in the Commons.
Official figures published in August showed UK net migration - the difference between those entering and leaving - went up by more than 38% to 243,000 in 2013-14, with EU citizens accounting for two-thirds of the growth.
It has been reported the UK could seek to apply an "emergency brake" to stop EU migration after it reached a certain level or to limit the number of National Insurance numbers issued to new arrivals.
Speaking to the Sunday Times, German Chancellor Angela Merkel appeared to rule this out, saying there could not be "tampering" with the EU principle of free movement of people.
This prompted Mr Fallon to say parts of the UK felt "under siege" from migrant workers and overseas benefit claimants and that some could be "swamped".
Labour and the Liberal Democrats criticised the comments and a government source said: "He accepts he should have chosen his words better."
Mr Fallon confirmed this, telling BBC Radio 5 live: "I misspoke yesterday, I used words I wouldn't normally have used."
Speaking to the BBC News Channel, he added: "I was a little careless in the words I used yesterday. I accept that.
"What I meant was there's huge pressure on the system now, on schools, housing and social services."
For Labour, shadow foreign secretary Douglas Alexander said Mr Fallon's claim that communities were being "swamped" reflected "the desperation of the Conservative Party". "You have got to be responsible always in the language that you use around issues of immigration," he told Sky News.
Lib Dem Energy Secretary Ed Davey said: "I've worked with Michael Fallon and I really respect him but I think those comments are more based on the Conservative concerns of the UKIP threat in the Rochester by-election than they are based on the facts."
Mr Cameron also faced questions over the European Arrest Warrant, ensuring the quicker transfer of criminal suspects between EU nations, when he speaks.
Some Conservative MPs argue the UK should reject it, arguing it undermines the national sovereignty over justice, but the government says it is vital to ensure terrorists, sexual offenders and murderers are dealt with promptly.
The Commons is expected to vote on the issue at about the time of the Rochester and Strood by-election, with Mr Cameron possibly facing a rebellion from his own MPs.