David Cameron has said Fifa president Sepp Blatter must resign, adding "the sooner that happens the better".
His comments came before Mr Blatter was re-elected after refusing to withdraw from the contest, despite the arrest of senior colleagues on major corruption charges.
But the PM said Fifa could only rebuild its credibility if Mr Blatter left.
It comes as the UK's Serious Fraud Office (SFO) said it was looking at material relating to the allegations.
Seven senior Fifa officials were arrested at a hotel in Switzerland on Wednesday.
The US Justice Department (DOJ) has charged 14 people with racketeering, wire fraud and money laundering conspiracy.
Mr Cameron's comments came ahead of the vote for the presidency of Fifa in Zurich, Switzerland.
Despite widespread criticism, Fifa president Sepp Blatter was re-elected for a fifth term.
His only rival, Jordan's Prince Ali bin al-Hussein, withdrew after the first round of voting in which neither candidate managed to secure the two-thirds majority needed.
Speaking in Berlin, Mr Cameron said: "You cannot have accusations of corruption at this level and on this scale in this organisation and pretend that the person currently leading it is the right person to take it forward.
"That cannot be the case. Frankly, what we've seen is the ugly side of the beautiful game and he should go.
"And the sooner that happens the better - the faster that organisation can start to rebuild its credibility, which is going to be so important because so many people around the world want to see this game properly managed, properly looked after, so we can all enjoy the World Cups of the future."
'Guided' on boycott
Mr Cameron, who is holding EU reform talks with German Chancellor Angela Merkel, said it was "unthinkable" that Mr Blatter should stay.
He also said he would be "guided" by the Football Association (FA) chairman, Greg Dyke, over the question of whether the bids for the 2018 and 2022 World Cups should be reopened.
Former culture minister David Lammy, who plans to run for London mayor in 2016, said Qatar was "not a fit venue" for the World Cup following the alleged corruption.
He said: "I want to see David Cameron and [London Mayor] Boris Johnson saying this isn't OK and offering Britain as an alternative venue.
"Let's put a serious offer on the table for the home of football - Britain - to host the 2022 World Cup."
Earlier on Friday, Mr Dyke suggested that he would not rule out a boycott of the major football tournaments if Mr Blatter remained at the helm of Fifa.
The Union of European Football Associations (Uefa), Europe's governing body, may consider boycotting future World Cups if Mr Blatter wins Friday's vote, according to Scottish FA chief executive Stewart Regan.
Meanwhile, the SFO said it was actively assessing "material in its possession" relating to Fifa corruption allegations.
It said it had "made plain that it stands ready to assist ongoing international criminal investigations".
It said its probe had been ongoing for months, before Wednesday's indictments.
The SFO said it was not willing to comment further on the nature of material in its possession or current activity in this area.
It added that it had not launched a formal criminal investigation. But it said there were two ways in which it could get involved - if the SFO opened a formal criminal investigation itself or provided "mutual legal assistance" to overseas law enforcement.
It also says it has made plain to both US and Swiss authorities that it stands ready to assist.
"There are currently international criminal investigations into allegations of corruption at Fifa. As such it would be inappropriate for the SFO to comment on the matter at this time," it said.
In November the SFO asked for anyone with information regarding alleged corruption in the bidding for the 2018 and 2022 World Cups to come forward.
However, it will not now say if its current actions relate to either of the above tournaments.