US President Donald Trump hit back at business leaders on Tuesday as executives tried to distance themselves from the administration.
Mr Trump is under fire for being late to condemn white supremacists and neo-Nazis involved in a violent rally.
But Mr Trump said: "They're leaving out of embarrassment, because they make their products outside."
Shortly after Mr Trump's comments, a fifth group stepped down from a White House business panel.
Those who have quit the manufacturing council in recent days include Kenneth Frazier of Merck, Kevin Plank of Under Armour, Brian Krzanich of Intel and Scott Paul, the president of business group the Alliance for American Manufacturing.
On Tuesday evening, after a combative news conference in which Mr Trump defended his original statement that violence came from "many sides", Richard Trumka, president of the AFL-CIO labour group, also said he and Thea Lee, another leader of the organisation, would no longer participate.
"It's clear that President Trump's manufacturing council was never an effective means for delivering real policy that lifts working families and his remarks today were the last straw," he said.
As calls mount for corporations to respond, other firms participating on White House panels have issued statements condemning the violence.
Walmart, which typically avoids political controversy, shared a statement from its chief executive that said Mr Trump "missed a critical opportunity to help bring our country together by unequivocally rejecting the appalling actions of white supremacists".
However, Walmart boss Doug McMillon did not say he would step down from the panel.
Shannon Coulter, who co-founded the #grabyourwallet boycott against companies that do business with Mr Trump, said recent events have added momentum to the campaign.
"Charlottesville has definitely escalated the issue of associating oneself with the Trumps," she told the BBC. "I think it's increasingly clear to CEOs on his councils that the Trump name and identity is toxic and that for the sake of their brands they need to get away from it as quickly as possible."
Kenneth Frazier, the head of drugs giant Merck, led the walkout from the White House manufacturing council on Monday morning.
One of only a handful of black leaders of Fortune 500 companies, Mr Frazier said he would no longer participate, calling it a matter of "personal conscience".
Mr Frazier said: "I feel a responsibility to take a stand against intolerance and extremism."
"America's leaders must honour our fundamental views by clearly rejecting expressions of hatred, bigotry and group supremacy, which run counter to the American ideal that all people are created equal."
In response, Mr Trump tweeted that Mr Frazier would now have "more time to lower rip-off drug prices".
Now that Ken Frazier of Merck Pharma has resigned from President's Manufacturing Council,he will have more time to LOWER RIPOFF DRUG PRICES!— Donald J. Trump (@realDonaldTrump) August 14, 2017
Mr Frazier's decision sparked calls from the public for other leaders involved in Mr Trump's panels to follow suit.
Kevin Plank, the chief executive of sports apparel company Under Armour, said he was resigning on Monday night. His decision came after he faced backlash from shoppers - and some Under Armour-sponsored athletes - earlier this year when he praised Mr Trump's pro-business views.
Intel boss Brian Krzanich also said on Monday that he would resign, followed by Scott Paul of the manufacturing alliance on Tuesday.
Mr Trump dismissed the resignations, saying those companies relied on overseas manufacturing.
"They're not taking their job seriously as it pertains to this country," he said. "We want jobs, manufacturing in this country."
Earlier business response
Many executives, including those at companies such as Campbell Soup Co and General Electric, said they feel it is important to remain involved.
"We must engage if we hope to change the world and those who lead it," Alex Gorsky, chairman and chief executive of Johnson & Johnson, said in a statement.
But the resignations this week add to Mr Trump's growing alienation from the business community, which he had expected to claim as an ally.
Former Uber chief executive Travis Kalanick left a business advisory council in February over the Trump administration's immigration policies.
Tesla's chief executive Elon Musk and Walt Disney's chief executive Robert Iger left the President's Strategic and Policy Forum in June, after Mr Trump said he would withdraw from the Paris climate accord.
Mr Musk also left the manufacturing council.
Rashad Robinson is executive director of Color of Change, which is among the groups that have brought pressure on corporations.
He said he hoped it would help make it clear that Mr Trump's attitude toward white supremacists and neo-Nazis was unacceptable.
"The more desertions and defections, the more isolated this administration is, the less mainstream connections that this administration has, the more powerful this message is to every day Americans about how out of step what's happening at 1600 Pennsylvania Avenue is and why we need a change, " he said.
'Having to choose'
Companies that cut ties with the White House are likely to face costs, said Jiekun Huang, a professor of finance at the University of Illinois at Urbana-Champaign.
He is co-author of a study that linked higher stock prices to White House visits, based on records from the Obama administration. An initial review of the first six months of the Trump administration showed a similar effect, he said.
But the risk of losing access to discuss regulations or contracts must be weighed against the risk of alienating employees and consumers, said Michael Maslansky, head of Maslansky + Partners, a language strategy firm that has advised major companies.
"The era of the fence-sitter corporation is over," he said.
"If you're silent about an issue, then each side will assume you're on the wrong side. You end up really having to choose."