How hard can it really be to herd cats?

Ask Daniel Mills, professor of veterinary behavioural medicine at the University of Lincoln, UK. In a recent study, Mills and his colleague Alice Potter demonstrated that cats are more autonomous and solitary than dogs. Carrying out the research for the project was as difficult as the cat's reputation might suggest.

"They are challenging if you want them to do certain things in a certain way," says Mills. "They tend to do their own thing."

Cat owners everywhere will sympathise. But why exactly are cats so reluctant to cooperate, either with each other or with a human? Or to flip the question around, why are so many other animals – wild and domestic – willing team players?

Group living is very common in nature. Birds flock, wildebeest herd, fish school. Predators often hunt together too, of course. Even the domestic cat's relative, the lion, lives in a pride.

Many group-living animals live by the "it takes a village" maxim

For prey species there is an obvious "safety in numbers" element to the strategy. "It's called the dilution effect," says biologist Craig Packer of the University of Minnesota in St Paul. "A predator can only take one, and if there's a hundred of you that reduces your chances down to 1%. Whereas if you're by yourself, you'll be chosen 100% of the time."

Animals in herds also benefit from the "many eyes" effect: the larger the prey group, the more likely they are to detect an approaching predator. "And the earlier you can detect the predator, the more time you have to initiate avoidance behaviour," says Jens Krause of Humboldt University in Berlin, Germany.

This collective vigilance brings other advantages. Individuals can afford to spend more time and energy looking for food.

It is not all about avoiding predators, though. Animals that socialise in groups do not have to roam around for a suitable mate, which is a problem for solitary species that cover large territories. It is clearly much easier just to look to the herd for a date.

As both bats and emperor penguins can testify, it can be just plain warmer when you huddle up to friends

And once they have produced their offspring, many group-living animals live by the "it takes a village" maxim: adults may work together to protect or feed the young.

In many bird species, such as the Arabian babblers of Israel, young birds remain in family groups until they are ready to mate. They dance in groups, bathe in groups, and even present each other with gifts.

There are also energy savings to be made from group living. Birds that flock together or fish that school move more efficiently than solitary fliers and swimmers.

It is the same principle that Tour de France cyclists use when they form a peloton. "The ones that are further back, they don't have to invest quite as much energy for the same speed of locomotion," says Krause.

Finally, as both bats and emperor penguins can testify, it can be just plain warmer when you huddle up to friends.

With so many benefits, it might seem surprising that any animal would shun its peers. But, as domestic cats show, group living is not for everyone. For some animals, sharing food just is not worth the benefits of being in a group.

"When you're feeding together, there's a point at which feeding with other individuals in close proximity reduces your intake," says John Fryxell, an integrative biologist at the University of Guelph in Canada.

You don't want to be around somebody else when they've just caught a mouse, because they're going to eat it whole

A key factor then becomes whether each animal has sufficient food, which in turn depends on how much food each animal needs. And cats have expensive tastes. For instance, a leopard will chew its way through 50lb (23kg) of meat every few days. Typically, for wild cats, the competition for food is fierce, so leopards live and hunt alone.

There is one exception to this solitary cat rule: lions. For lions it is a territorial deal, says Packer, who has spent nearly five decades studying African lions. Certain locales in the savannah grasslands offer perfect ambush spots for prey, so controlling that prime real estate offers a significant survival advantage. "It imposes sociality because you have to have teams to dominate your local neighbourhood, exclude competing teams, and the biggest team wins," Packer says.

What makes this happy hamlet possible is that a single lion kill – a wildebeest or zebra – is large enough to feed several females at once. "So the size of the prey allows them to live in groups, but it's the geography that really drives them toward living in groups," Packer says.

This is not a situation domestic cats find themselves in, because they hunt small animals. "You don't want to be around somebody else when they've just caught a mouse, because they're going to eat it whole," Packer says. "It's gone. There is not enough food to share."

This sort of economic reasoning is so deeply engrained in cat behaviour, it seems unlikely that even domestication could have altered this fundamental preference for solitude.

This is doubly true when you consider that humans did not domesticate cats. Instead, in true cat style, cats domesticated themselves.

If they do accidentally come face to face, the hackles rise and the claws come out

All domesticated cats are descended from Middle Eastern wildcats (Felis silvestris), the "cat of the woods". Humans did not coax those early cats out of the woods; the cats invited themselves into our grain storehouses, where an abundance of mice fed unchecked. Gate-crashing this mouse party marked the start of a truly symbiotic relationship. The cats loved the well-stocked storehouses, and the people appreciated the pest control.

Domestic cats are not entirely antisocial. But their sociability – to one another or to their human owners – is handled entirely on their terms.

"They retain a large degree of independence and approach, or stay close to us, only when they want to," says Dennis Turner, a cat expert and animal behaviourist at the Institute for Applied Ethology and Animal Psychology in Horgen, Switzerland.

"Cats have evolved lots of mechanisms to keep themselves apart, which aren't exactly conducive to herding," says Mills. Cats spray their territory to help avoid awkward meetings with each other. If they do accidentally come face to face, the hackles rise and the claws come out.

If cats will not stop fighting, they are unlikely to start herding.

In some circumstances it can appear that domestic cats have embraced group living; for instance, a colony living in a barn. But do not be fooled, says Fryxell.

"They're very loose aggregations and they don't have any real group identity," he says. "They just have a common place they come to keep their kittens."

In keeping with their solitary, uncooperative reputation, cats turned out to be neurotic, impulsive and resistant to being ordered around

In fact, even in the face of extreme danger, which often brings animals together to form a defensive unit, it is unlikely cats would cooperate. "It's just not something that they typically do when they're threatened," says Monique Udell, a biologist at Oregon State University. Cats just do not believe in strength in numbers.

All of this helps explain why cats have a reputation for being impossible to herd. Even so, there is some evidence that the domestic cat's disdain for group living could be weakening.

A study published in 2014 in the Journal of Comparative Psychology saw scientists probe the personality traits of domestic cats. In keeping with their solitary, uncooperative reputation, cats turned out to be neurotic, impulsive and resistant to being ordered around.

Interestingly, though, domestic cats might be a little less uncooperative than their wild relatives. When the researchers compared the domestic cat to four wild cats – Scottish wildcats, clouded leopards, snow leopards and African lions – the domestic cats proved to be most like those group-living lions in terms of their overarching personalities.

It is also fair to say that domestic cats have come a long way from their ancestors in terms of tolerating each other's company. Although groups of cats living in barns form loose aggregations at best, they still show a remarkable level of acceptance for each other's presence in such confined spaces.

In Rome, nearly 200 cats live side-by-side in the Colosseum, while on Aoshima Island in Japan cats outnumber people six to one. These colonies may not be cooperatives, but they are a far step from the domestic cat's solitary past.

Obviously, lions managed the feat, so it must be possible for the proper set of mutations to occur

Researchers, meanwhile, might find it easier to control the cats in their experiments by meeting them halfway.

When Udell performed her early experiments on cats, she encountered a lot of difficulties trying to motivate her subjects to engage in the task at hand. She had previously worked with dogs, which would willingly perform any task repeatedly for a food reward. But cats were not so easily pleased. Eventually, Udell started having success when she began giving cats the option of choosing what their reward was.

"I think that part of the challenge is just how much we understand about cats," she says. If scientists start to gain insights into the black box of the feline mind, herding by force could give way to convening cats using cunning coercion.

Much of animal behaviour – including an affinity or resistance to herding – lies deep in a creature's neural circuitry. It is difficult to ever imagine circumventing years of natural selection, Fryxell says.

"But who knows? Obviously, lions managed the feat, so it must be possible for the proper set of mutations to occur," he says. "And if they managed to pull off the trick, maybe [herding cats] is not that crazy an idea, after all."

Join over six million BBC Earth fans by liking us on Facebook, or follow us on Twitter and Instagram.

If you liked this story, sign up for the weekly features newsletter called "If You Only Read 6 Things This Week". A handpicked selection of stories from BBC Future, Earth, Culture, Capital, Travel and Autos, delivered to your inbox every Friday.