The Home Office has backed Baroness Butler-Sloss as the right person to lead an inquiry into allegations of historical child abuse, after claims about her over a previous review.
Phil Johnson, who was abused while a choirboy, claims she wanted to exclude some of his allegations in a bid to protect the Church of England.
He says she told him she "cared very much about the Church".
Baroness Butler-Sloss said she had never put institutions before victims.
The Rt Rev Peter Ball, who was bishop of Gloucester and bishop of Lewes in East Sussex, was charged with two counts of indecent assault and one of misconduct in a public office, following her investigation into abuse in the diocese of Chichester during the 1970s and 1980s.
A court heard in April that the 82-year-old retired bishop was too unwell to answer the allegations.
Mr Johnson's claims add to pressure on Baroness Butler-Sloss, who was appointed by Home Secretary Theresa May last week to head a review of how allegations of abuse linked to public institutions in the 1970s, 80s and 90s were handled.
The former High Court judge has faced calls to step down because of her establishment links.
Her detractors point to her late brother, Sir Michael Havers, having been attorney general at the time of the alleged organised abuse that is the subject of her review.
Labour MP Simon Danczuk said her position was tainted because of the connection, while Alison Millar, the lawyer who represents alleged victims of child abuse, said she doubted her clients would think Lady Butler-Sloss was the right person for the job.
Keith Vaz, Labour chairman of the Home Affairs Select Committee pointed out that while Lady Butler-Sloss was "distinguished", she was also a member of the House of Lords.
But the Home Office said it was backing her "unreservedly", adding that she was "beyond reproach".
"Baroness Butler-Sloss has had a long and distinguished career at the highest levels of this country's legal system," a spokesperson said.
"Her work leading the Cleveland child abuse inquiry and as president of the High Court's Family Division make her the perfect person to lead this important piece of work."
Mr Johnson, who suffered assaults by a number of clergymen when he was a choirboy in the Church of England Diocese in Chichester, said he felt during her review of Church abuse, Baroness Butler-Sloss had been "showing bias and wasn't being impartial".
At a meeting in her House of Lords office in 2011, Mr Johnson says: "She told me that she cared very much about the Church and seemed to be wanting to protect the Church's image."
Mr Johnson alleges that Baroness Butler-Sloss had told him if she included the bishop's name in her report it would distract from the more serious abuse of two priests.
But he also said that she "didn't want to generate any excessive negative publicity for the Church".
"She expressed that by saying that 'the press would love a bishop' and she didn't want to give the press that trophy."
Baroness Butler-Sloss said: "Throughout many years of public service I have always striven to be fair and compassionate, mindful of the very real suffering of those who have been victims of crime or other injustice.
"I have never put the reputation of any institution, including the Church of England, above the pursuit of justice for victims."
Mr Johnson accepts she passed on his allegation against the bishop but says he felt he had little choice but to agree with her decision not to include it in her report.