The Vikings used a lotion of goose poo. The ancient Greek medic Hippocrates believed the best cure for baldness was really pigeon droppings, which he mixed with horseradish, cumin and nettles. One 5,000 year-old Egyptian recipe suggested blending the burned prickles of a hedgehog immersed in oil with honey, alabaster, red ochre – oh, and fingernail scrapings – and slathering the concoction liberally over the affected area.

For as long as men have had access to mirrors, they’ve been fretting about their scalps getting lonely. It was a particular obsession of Julius Caesar, who tried everything to get his hair back; the wreath of laurels he wore was less a nod to Roman tradition than an attempt at covering up his shiny pate.

By the time he met Cleopatra, he was almost completely bald. In a last ditch attempt to save his mop, she lovingly recommended a home remedy of ground-up mice, horse teeth and bear grease.

Alas, it didn’t work. He lost his hair like many great men before and since, including Socrates, Napoleon, Aristotle, Gandhi, Darwin, Churchill, Shakespeare and Hippocrates – who, despite the pigeon droppings, was so bald he even has a type of baldness named after him. Eventually Caesar began growing his hair longer at the back and combing these strands forwards across his head, a technique which was optimistically described as “illusion styling”. Now it’s known as the comb-over.

Thousands of years later, we’ve moved on from garlands and revolting concoctions to expensive creams, tonics and shampoos, and last resorts of toupes, pills and surgery. Today you can attend a hair loss clinic, sign up for hair-loss counselling and it’s not unusual to see adverts telling balding men to “see their doctor”. Papers discuss balding in epidemic terms, meanwhile the phenomenon even has a new scientific-sounding name, “androgenic alopecia”. If you didn’t know otherwise, you might think it was a medical condition.

Accordingly, across the globe, we spend $3.5bn (£2.7bn) on baldness cures every year. That’s more than the entire national budget of Macedonia or, as Bill Gates pointed out last year, significantly more than we spend on the control of malaria (just $200m (£154m) per year).

And what today’s treatments lose out on in ingredients they make up for in unpleasant side-effects. Sales of the anti-hair-loss drug Propecia, which has been linked to impotence, hit a record high of $264m (£204m) in 2014. Hair transplants, meanwhile, are notoriously bloody and have been known to make grown men cry. According to a 2009 survey by the International Society of Hair Restoration Surgery, almost 60% of men would rather have a full head of hair than money or friends.

Have we got it all wrong?

There’s mounting evidence that bare heads aren’t a spectacular evolutionary accident after all. Bald men are seen as more intelligent, dominant and high status; their shiny scalps may help them to seduce women or even save lives. 

Before we can get to grips with what makes balding so great, first we need to set the record straight. 

Contrary to popular wisdom – and the existence of super-macho baldies such as Bruce Willis – the propensity to lose your hair doesn’t make you any more of a man. Bald men are no more virile and they don’t have higher testosterone levels, though they do tend to have hairier arms, legs and chests. Perhaps most surprisingly, bald men don’t actually have any fewer hairs on their heads.

So how does it happen?

In 1897, a wave of panic rippled across the globe after a French dermatologist announced he had discovered the true culprit: a microbe

For all the hysteria surrounding baldness, we have a long history of getting it wrong. Aristotle thought it was caused by sex. In ancient Roman times, an epidemic of smooth scalps in the military was blamed on the heavy metal helmets worn by soldiers. Later theories included “dryness of the brain” – which was thought to pull the head away from the hair by making the brain shrink – air pollution or, ironically, the wrong haircut.

Back in 1897, a wave of panic rippled across the globe after a French dermatologist announced he had discovered the true culprit: a microbe. Barbers and medical journals leapt to action, announcing that combs should be boiled regularly and under no circumstances should members of bald families use any combs or brushes other than their own.

We now know that baldness is caused by a potent break-down product of testosterone, dihydrotestosterone (DHT). In the womb, the hormone plays an important role in the development of male genitalia. In susceptible adults, it makes hair follicles shrink. As DHT works its magic the long, so-called “terminal” hairs on a man’s head are transformed into short, soft “vellus” hairs like those on the heads of babies.

What sets those with bare heads apart is the sensitivity of their hair follicles, which is inherited from their mothers

And since it’s a product of testosterone, you might think more testosterone would equal more DHT and more withered hairs. In fact you only need a tiny amount for balding to begin. What sets those with bare heads apart is the sensitivity of their hair follicles, which is inherited from their mothers.

The fact that it’s inherited is crucial. By the time they reach their 30th birthday – long before the end of their reproductive lifespan – 25-30% of men have some degree of hair loss. Not only that, but it happens all over the world, in every single ethnic group. If being bald was so bad, it would have died out. The fact it is so common might suggest it is useful, but how? And if so, why does it only happen to men?

“In general in nature when males have something that the females don’t it means that characteristic is acting as a signal,” says Frank Muscarella, a psychologist at Barry University. Back in the 1990s, this got him thinking.

Most of these “sexually dimorphic” features also have something else in common. “They are usually associated with dominance and more reproductive opportunities,” says Muscarella. In other words, baldness might be the human equivalent of a male peacock’s ornate, brightly coloured tail. It might have evolved because it is attractive to women.

Previous studies had shown that women do not find bald men sexy, but this is likely to be because bald men tend to be old, and – not surprisingly – women don’t find old age appealing. “We know that women are attracted to men with high social status, so even if it’s not physically attractive it may embed a kind of non-physical attractiveness,” says Muscarella.

Before you ask, Muscarella doesn’t have a vested interest in this theory. “No actually I’m not bald, I have a very good head of hair,” he says.

In 2004, he decided to investigate on behalf of those who are less fortunate. To eliminate factors which might skew his results, Muscarella knew he’d need to improvise some bald men himself. But he knew he couldn’t simply photograph some men and edit their hair out.

Instead Muscarella roped in a hairdresser friend – and they took a trip to the local wig shop. “I asked him to cut the hair on the wigs so that one looked like a full head of hair for a man, one looked like it was receding and one looked like it was bald,” he says. Men who go bald naturally tend to retain some hair over their ears and around the back of the head, so they needed a wig even for the bald look.

After his friend had finished hacking the wigs to shape, Muscarella bought three plastic skullcaps and attached the hair with Velcro. Then they asked six unfortunate students to try them on and took some photos. “Of course they looked absolutely horrible,” he says.

Luckily his co-author had just acquired some very, very primitive photo editing software and so, pixel by pixel, they were able to erase the strong lines between the skullcap and the forehead and make the pictures look relatively normal.

Then they put their images to the test. He showed the images to 101 male and 101 female psychology students and asked them to rate the men’s attractiveness and aspects of their personality.

The bald men were consistently rated as more intelligent, influential, knowledgeable, well educated, high social status, honest and helpful

While the bald and balding men were not considered as physically attractive as the other men, one category of scores was far higher. The men were consistently rated as more intelligent, influential, knowledgeable, well-educated, high social status, honest and helpful – traits collectively known as social maturity. 

He speculates that baldness may have evolved as a signal of high social status – something some women find irresistible. Intriguingly, bald men were also viewed as significantly less aggressive. “If you think about it, early human males would have been running around completely naked and completely hairy, so you can imagine a big shaggy head of hair, a big bushy moustache – all kinds of body hair – they would have been very kind of threatening looking,” says Muscarella. Going bald may have been a way to separate mature, high-status men from hostile adolescents.

If that’s the case, it’s possible we’ve been hi-jacking this natural signal for years already. As Muscarella points out, the shaven look has been a hit with philosophers, teachers and priests for centuries. Christian monks have taken this one step further – not only shaving their heads, but doing so in a way which directly mimics the way men go bald.

The finding is backed up by numerous other studies. Across the globe, from workers on a remote sugar cane plantation in Brazil to Zambian high school students, balding men are overwhelmingly viewed as more dominant. It even works on men who have simply shaved all their hair off.

There’s even some tantalising – albeit controversial – evidence that bald heads may be life-saving.

For a long time, the opposite was true. It’s long been known that men who can’t produce DHT, such as those who have been castrated, have healthy, flowing locks for as long as they live. Intriguingly, there are also no recorded cases of prostate cancer in this group.

DHT is responsible for the growth of the prostate gland in babies, so it makes a lot of sense that it would also contribute to the growth of tumours in adulthood. Prostate cancer and baldness may be underpinned by a shared hormonal sensitivity which runs in the family. This was confirmed earlier this year with the discovery that balding men are more likely to develop aggressive prostate cancer, which is responsible for around 300,000 deaths every year.

It doesn’t sound like much of a silver lining – but there’s a twist. Low levels of vitamin D – which the body can only produce when it’s exposed to sunlight – is another known risk factor for prostate cancer. And as any bald man will tell you, they get a lot more sun exposure than the rest of us. Did baldness evolve to mitigate some of the deadly effects of DHT?

“Tens of thousands of years ago in Europe it might have helped those individuals to receive more UV radiation and produce more vitamin D,” says Peter Kabai from István University in Hungary, who struck upon the idea when he began going bald. This would also explain why women don’t go bald, since they don’t have a prostate.  

Men who develop a receding hairline and bald spots before the age of 30 are up to 45% less likely to develop prostate cancer later on

The evidence is racking up. Men who work outdoors have better odds against those who spend more time inside. That’s also true of those who have a tan, who were sunburnt as children, who live in warmer climes or who take more holidays abroad. The effect is so powerful, even the season in which you’re diagnosed makes a difference: those diagnosed in the summer are less likely to die of their cancer.

“This all relates to vitamin D – in which most people are deficient,” says Kabai. The final strand of proof comes from a clinical trial published last year. Thirty seven men with prostate cancer were either given a vitamin D supplement (nearly seven times the recommended daily allowance) or a placebo.

Sixty days later, their prostates were removed. In the group which had been taking vitamin D, their tumours had shrunk. In the group which hadn’t, they had got worse. The supplement also changed how key genes were expressed – turning off those involved in inflammation, which is known to contribute to the development of cancer.

In other words, balding men may be more susceptible to prostate cancer despite the fact that they are losing their hair, not because of it; the baldness itself may be helping mitigate some of the risk.

It could also clear up why the evidence is so confusing: a previous study found that men who developed a receding hairline and bald spots before the age of 30 were up to 45% less likely to develop prostate cancer later on. “Some bald men might prefer to wear hat all the time, while others might not. That difference could be one source of ambiguities in such studies,” says Kabai.

So there you have it: going bald could help men get ahead, get the girl or get better. Perhaps it’s time to put the pigeon droppings away and give bare heads the respect they deserve.

Join 700,000+ Future fans by liking us on Facebook, or follow us on Twitter, Google+, LinkedIn and Instagram.

If you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Earth, Culture, Capital, Travel and Autos, delivered to your inbox every Friday.