The chancellor of Oxford University says there is a "bit of hypocrisy" over the calls for a statue of British imperialist Cecil Rhodes to be removed.
Lord Patten said there should be "an argument about far more fundamental issues which touch on Black Lives Matter" such as education and health.
Thousands peacefully protested to remove the statue on Tuesday as part of the Black Lives Matter movement.
City councillor Shaista Aziz said Lord Patten's response was "tone deaf".
Campaigners have said Rhodes, a 19th Century businessman and politician in southern Africa, represented white supremacy and is steeped in colonialism and racism.
Crowds chanted "take it down" outside Oriel College in High Street during the protest organised by the "Rhodes Must Fall" campaign.
Lord Patten, who has no control over the future of the statue, said: "There is a bit of hypocrisy... in Oxford taking money for 100 scholars a year, about a fifth of them from Africa, to come to Oxford, and then saying we want to throw the Rhodes statue... in the Thames."
He said his "own view" remained the same as one "expressed by Nelson Mandela at a celebration of the Rhodes Trust in 2003".
He said despite the "problems associated with Cecil Rhodes in history, if it was alright for Mandela, then I have to say it's pretty well alright for me".
"Almost every aspect of history you have to look at both sides and normally there is more than two sides," he said.
In his 2003 speech, Mandela paid tribute to Rhodes, saying he was "part of shaping what present day South Africa turned out to be".
He said his legacy was "controversial" but that he should still be "remembered by posterity" for his philanthropy.
Councillor Aziz, one of 26 who backed calls for the statue to be removed, said the chancellor's comments showed "how out of touch this institution is with the people of our city".
"This is Grade A+ whataboutery by a world famous institution that could take this opportunity to really engage with the residents of our city," she said.
Oriel College told the BBC there had been "no change" to its position on the statue. It previously said it "abhors racism and discrimination in all its forms".
The college decided not to remove the statue following protests in 2016 and has rebutted claims it was due to donors threatening to withdraw funding.
More than 150,000 people have signed an online petition calling for the statue to be removed.
Oxford University has not replied to the BBC's repeated requests for comment.
The Rhodes Must Fall campaign began in South Africa, where a Rhodes statue was removed before it was adopted in Oxford.
Rhodes was a member of Oriel College in the 1870s. He left money to the college on his death in 1902.
A scholarship programme in his name has been awarded to more than 8,000 overseas students, including former US president Bill Clinton.