Quinton Fortune says he understands Rio Ferdinand's anger with the football authorities given his lengthy ban for missing a drugs test in December 2003.
The Manchester United defender was banned by the Football Association for eight months and missed Euro 2004.
Ferdinand's former team-mate Fortune feels racial abuse is the bigger issue.
"In the last few months players have been banned for eight games or four games after racial incidents," the South African told BBC Sport.
"I don't condone missing drug tests but when you look at it from the outside something is not right. I think the racial abuse is stronger."
Ferdinand, 33, received his lengthy sanction when he and Fortune were team-mates, while Chelsea captain John Terry was handed a four-match ban for racially abusing Ferdinand's brother, Anton, at Loftus Road last October.
Prior to that, Liverpool's Luis Suarez received an eight-match suspension for racially abusing Ferdinand's current Manchester United team-mate Patrice Evra.
In a statement released by the Ferdinand brothers on Wednesday they said that despite their disappointment with the FA and Professional Footballers' Association for their handling of the Terry case, they would be "committed to working with football's existing organisations towards the betterment of the game and to achieve immediate action."
Fortune, 35, has suffered racial abuse throughout his career in South Africa, England, Spain and Italy and revealed at a Kick It Out event that he regrets not standing up to the perpetrators.
He says players who refused to wear Kick It Out T-shirts should break their silence too, or else they will become frustrated with themselves as Fortune has.
The former midfielder believes they are protesting because there is a lack of understanding from football authorities such as the FA.
"The people running the game don't understand what the current players are going through," he added.
"It's a different generation. They need to speak to [Reading striker] Jason Roberts or Rio and the other black players; sit them down and get their feelings and move forward.
"Making decisions without them builds up anger and you've seen the decisions that have been made over the last few weeks, it's been frustrating."
Fortune, who was at Old Trafford to discuss anti-Semitism, added: "I've learned from experience back home where waiting for the people at the top takes forever.
"Players know what's going on and the [authorities] need to speak to the black players and get their opinions because they are the ones who are going through it.
"It took 12 months [to deal with the Terry case]. That could have been dealt with quickly. People make mistakes, but we have to forgive them and move on and try and educate them instead of dragging it out and making that individual look so bad.
"He's probably just made a mistake. He's a human being, everyone makes mistakes but dragging it on makes it look worse than it is."