Kick It Out chairman and founder Lord Ouseley says he has never "abused or bullied anyone" after concerns were raised about staff welfare.
It was reported by the Daily Mail on Tuesday that 10 members of staff have left since the start of 2016 because of an alleged poor working environment.
The allegations are being investigated by the Charity Commission.
In a statement, Lord Ouseley said he had "no criticism to make of any staff who wish to be critical".
The 73-year-old announced last week that he would step down as chairman after 25 years in the role.
"I fully accept my role as leader and accept any failing which may have occurred," his statement read.
"However, that does not mean that I am going to accept unreasonable and unjustified media criticism.
"I have never, in my view, and would defy anyone to bring forward evidence to the contrary, abused, vilified, harassed or bullied anyone."
Lord Ouseley founded the football anti-discrimination charity in 1993.
A day after the Daily Mail's initial report, the newspaper claimed one of the staff to leave was threatened with legal action for suggesting they had been bullied.
On Thursday it reported a "serious incident" took place in 2017, but the charity's only response was to ban alcohol at future staff gatherings - a move the newspaper says angered members of staff who felt the charity's leadership was not taking it seriously enough.
In its initial response to the bullying claims, Kick It Out said nobody had raised the issue in an exit interview.
In its statement on the 2017 incident, it said a staff member was offered counselling and support and "we took all steps to help her".
It added: "The trustees were not informed until October of this year to safeguard the privacy required of the staff member concerned. She decided in September to inform some of the trustees herself."
Lord Ouseley's statement read: "I know that the Kick It Out senior staff gave their best endeavours and dedication to the affected staff member.
"I hold my hands up as guilty if I should have told my trustees and the Charity Commissioners."
Lord Ouseley is still one of the charity's eight trustees, alongside BBC pundit and former Stoke and Tottenham striker Garth Crooks, and the Premier League's head of policy Katherine Allen.