More than a third of black or minority ethnic workers (BME) have been subject to racism at work, a study by the TUC trade union body suggests.
Incidents included bullying, abuse, or being singled out for unfair treatment, a survey of 1,000 BME workers found.
Almost a fifth of the workers said they had been passed over for training or promotion, the TUC added.
Industry body the CBI said employers have a legal duty to protect their employees.
More than 43% of ethnic minority workers said they did not report discrimination to their employers, and 38% did not report bullying and harassment.
TUC general secretary Frances O'Grady said: "Racism still haunts the Britain workplace. Racist bullying, harassment and victimisation should have no place anywhere, least of all at work."
Dalvir [not his real name], a worker in the Midlands, complained about a racist joke made by his manager, and work became more difficult.
"After I submitted the complaint, there was a campaign to remove me. They increased my supervision and claimed I was absent from work, even though making home visits was part of my job.
"This took a toll on my health, and I had to take a few months off. That really slowed any career progress. It's a tough place for black people to work.
"Recently a black woman applied for a senior role. She was by far the best candidate for the job but it was given to someone else because they knew the manager. It's often hard to prove that it's racism, but I think most of us can spot a pattern."
Neil Carberry, CBI managing director for people, said: "All businesses should adopt a clear policy of zero tolerance, regardless of who the perpetrator is, and work with staff to build a culture that ensures people feel confident to speak out.
"Firms have a clear legal duty to protect their workers under both employment rules and the Health and Safety at Work Act."
Employers should intervene if any employee expresses racist views, according to conciliation service Acas.
Racism at work could be an employment law and a criminal law matter - so in some cases, employers and the police should handle a complaint, Acas added.
Acas adviser Tom Neil said companies should investigate incidents properly, as well as having an open workplace that celebrates cultural diversity.
A sample of 1,003 BME workers were polled online and asked to comment on their experiences over the last five years.