The sackings of Chris Powell and Chris Ramsey were "disheartening", says anti-racism campaigner Troy Townsend.
"To lose both managers sends out a really strong message to those aspiring to get on this journey," said Townsend.
But former Chelsea defender Frank Sinclair disagreed and called Townsend's claims "damaging".
Last week the Football Association's newly appointed BME boss Wayne Allison said he wanted to increase the number of black coaches.
But the sackings of both Powell and Ramsey within the space of a few hours on Tuesday left numbers at a low and could discourage black coaches, according to Townsend.
"People will look and think 'Is it worth it? Is it because of the colour of their skin? Or is it because they are bad managers?," Townsend told BBC Radio 5 live.
"I know people will say results are not right but there is a lot more to it than that.
"Today is disheartening without a doubt."
Chris Hughton (Brighton), Jimmy Floyd Hasselbaink (Burton Albion), Keith Curle (Carlisle) and Ricardo Moniz (Notts County) are now the only managers from BME backgrounds in the Football League after Wednesday's sackings.
Townsend - education and development manager at Kick It Out, as well as the father of Tottenham winger Andros - said the loss of both Powell and Ramsey from the Championship sides was "a really disappointing day for BME people who are looking to be part of the game at the highest level".
He told BBC Radio 5 live: "We are talking about the under-representation of black managers in the English game.
"We have 25% BME players in this country. There are less than 4% in terms of major coaching roles. The stats don't stack up."
'Totally out of order'
Sinclair was recently coaching Brackley Town in the National League North and says he "totally disagrees" with Townsend's claims.
The 43-year-old says that he has twice recently come close to a job and believes there are opportunities for black coaches in England.
"They [Powell and Ramsey] are both terrific lads who have given me advice in the past, and you're disappointed that they lost their jobs, but you've got to be realistic," he told BBC Radio Northampton.
"To make a statement that more or less says 'is it because of their colour that they've been sacked?' is something that I had to make a response to because I totally disagree.
"It can be damaging. I don't question a chairman's integrity.
"I've been very close to getting a couple of jobs recently, and out of 35-40 applicants, I've got to the final two twice. There's encouragement that if I keep doing the right things, then I'll get the opportunity to manage at the level that I want to.
"[Townsend's comments] are damaging. People could read that and think: 'If I take on a black manager is he going to turn around if it doesn't go well and say 'the reason I got sacked is because of the colour of my skin'?
"That is out of order and you can't have statements made like that."
Huddersfield said that the dismissal of Powell "signals a change of direction by the club and a new approach", while QPR director of football Les Ferdinand said that replacing Ramsey was in the "best interests of the club".