The football authorities must act to end the "sub-human" conditions of migrant labourers working on 2022 World Cup facilities in Qatar, Labour says.
Jim Murphy, who recently visited Qatar, said some workers had been lured to the country under false pretences.
The shadow international development secretary said he met migrant workers who then had their passports taken away so they could not leave.
The English and Scottish football associations have declined to comment.
Concerns have previously been raised by labour groups after dozens of migrant workers reportedly died working on construction projects in the country.
Following pressure from football's world governing body Fifa, organisers of the 2022 tournament developed a Workers' Charter in an attempt to protect the rights of migrant employees.
But Mr Murphy has now called for more pressure from football authorities after witnessing conditions first hand during a visit with the the International Trade Union Confederation (ITUC).
He said that many migrant workers had signed contracts in countries such as Kenya and Bangladesh in which they were "promised wages they couldn't earn at home", but when they arrived in Qatar their contracts were "torn up".
"Many of them have their passports seized from them by their employers" and were left "living and toiling in conditions that are sub-human", he said.
Mr Murphy, who met organisers of the tournament during his visit, said he had been given assurances there would be changes to the controversial sponsorship system known as Kafala.
The ITUC says that under the system, migrant workers are not allowed to end unfair employment contracts or change employers.
Mr Murphy said: "I think we all have to, through the football authorities in this country and beyond, make sure that those promises that were given to me are now delivered on."
He said the sport would be "embarrassed" and "ashamed" forever "unless Fifa and the football authorities act on workers' rights.