From fireside folk tales to Netflix dramas, narratives are essential to every society – and evolutionary theorists are now trying to figure out why, writes David Robson.

It sounds like the perfect summer blockbuster.

A handsome king is blessed with superhuman strength, but his insufferable arrogance means that he threatens to wreak havoc on his kingdom. Enter a down-to-earth wayfarer who challenges him to fight. The king ends the battle chastened, and the two heroes become fast friends and embark on a series of dangerous quests across the kingdom.

The fact that this tale is still being read today is itself remarkable. It is the Epic of Gilgamesh, engraved on ancient Babylonian tablets 4,000 years ago, making it the oldest surviving work of great literature. We can assume that the story was enormously popular at the time, given that later iterations of the poem can be found over the next millennium.

More like this:

- The fiction that changed history

An ancient poem that still sums up war

Why The Handmaid’s Tale is so relevant today

What is even more astonishing is the fact that it is read and enjoyed today, and that so many of its basic elements – including its heart-warming ‘bromance’ – can be found in so many of the popular stories that have come since.

Such common features are now a primary interest of scholars specialising in ‘literary Darwinism’, who are asking what exactly makes a good story, and the evolutionary reasons that certain narratives – from Homer’s Odyssey to Harry Potter – have such popular appeal.

Escapism?

Although we have no firm evidence of storytelling before the advent of writing, we can assume that narratives have been central to human life for thousands of years. The cave paintings in sites like Chauvet and Lascaux in France from 30,000 years ago appear to depict dramatic scenes that were probably accompanied by oral storytelling.

“If you look across the cave, there will be a swathe of different images and there often seems to be a narration relating to a hunting expedition,” says Daniel Kruger at the University of Michigan – narratives that may have contained important lessons for the group. Some tales from the last Ice Age may even linger today (see sidebar: What’s the oldest story?).

Today, we may not gather around the camp fire, but the average adult is still thought to spend at least 6% of the waking day engrossed in fictional stories on our various screens.

The more people read fiction, the easier they find it to empathise with other people

From an evolutionary point of view, that would be an awful lot of time and energy to expend on pure escapism, but psychologists and literary theorists have now identified many potential benefits to this fiction addiction. One common idea is that storytelling is a form of cognitive play that hones our minds, allowing us to simulate the world around us and imagine different strategies, particularly in social situations. “It teaches us about other people and it’s a practice in empathy and theory of mind,” says Joseph Carroll at the University of Missouri-St Louis.

Providing some evidence for this theory, brain scans have shown that reading or hearing stories activates various areas of the cortex that are known to be involved in social and emotional processing, and the more people read fiction, the easier they find it to empathise with other people.

Palaeolithic politics

Crucially, evolutionary psychologists believe that our prehistoric preoccupations still shape the form of the stories we enjoy. As humans evolved to live in bigger societies, for instance, we needed to learn how to cooperate, without being a ‘free rider’ who takes too much and gives nothing, or overbearing individuals abusing their dominance to the detriment of the group’s welfare. Our capacity for storytelling – and the tales we tell – may have therefore also evolved as a way of communicating the right social norms. “The lesson is to resist tyranny and don’t become a tyrant yourself,” Kruger said.

Along these lines, various studies have identified cooperation as a core theme in popular narratives across the world. The anthropologist Daniel Smith of University College London recently visited 18 groups of hunter-gatherers of the Philippines. He found nearly 80% of their tales concerned moral decision making and social dilemmas (as opposed to stories about, say, nature). Crucially, this then appeared to translate to their real-life behaviour; the groups that appeared to invest the most in storytelling also proved to be the most cooperative during various experimental tasks – exactly as the evolutionary theory would suggest.

The Epic of Gilgamesh provides one example from ancient literature. At the start of the tale the King Gilgamesh may appear to be the perfect hero in terms of his physical strength and courage, but he is also an arrogant tyrant who abuses his power, using his droits to seigneur to sleep with any woman who takes his fancy, and it is only after he is challenged by the stranger Enkidu that he ultimately learns the value of cooperation and friendship. The message for the audience should have been loud and clear: if even the heroic king has to respect others, so do you.

What’s the oldest story?

Although we have no firm evidence, it’s possible that some tales we still read today may have their origins in deep prehistory. Daniel Kruger points out that tales such as the Epic of Gilgamesh, and the Book of Genesis in the Old Testament, contain details of a mythical flood that may tap into lingering cultural memories of real, geological events in the Middle East from the end of the last Ice Age.

Indigenous people on the isle of Flores in Indonesia, meanwhile, have long had myths of the Ebu Gogo – short, hobbit-like creatures without language, which appear to relate to archaeological remains of a human sub-species that overlapped with the Homo sapiens population before going extinct more than 10,000 years ago. “The locals actually have stories of these little people who couldn’t really use language, but if you said something to them, they would repeat it. And that amazes me that a story like that could persist for literally tens of thousands of years.” All of which demonstrates another important purpose of storytelling – to offer a collective memory of times long past.

By mapping the spread of oral folktales across different cultural groups in Europe and Asia, some anthropologists have also estimated that certain folktales – such as the Faustian story of The Smith and the Devil – may have arrived with the first Indo-European settlers more than 6,000 years ago, who then spread out and conquered the continent, bringing their fiction with them.

In his book On the Origin of Stories, Brian Boyd of the University of Auckland describes how these themes are also evident in Homer’s Odyssey. As Penelope waits for Odysseus’s return, her suitors spend all day eating and drinking at her home. When he finally arrives in the guise of a poor beggar, however, they begrudge offering him any shelter (in his own home!). They ultimately get their comeuppance as Odysseus removes his disguise and wreaks a bloody revenge.

You might assume that our interest in cooperation would have dwindled with the increasing individualism of the Industrial Revolution, but Kruger and Carroll have found that these themes were still prevalent in some of the most beloved British novels from the 19th and early 20th Centuries.

Asking a panel of readers to rate the principal characters in more than 200 novels (beginning with Jane Austen and ending with EM Forster), the researchers found that the antagonists’ major flaw was most often a quest for social dominance at the expense of others or an abuse of their existing power, while the protagonists appeared to be less individualistic and ambitious.

Consider Jane Austen’s Pride and Prejudice. The conniving and catty Miss Bingley aims to increase her station by cosying up to the rich-but-arrogant Mr Darcy and establishing a match between her brother and Darcy’s sister – while also looking down on anyone of a lower social standing. The heroine Elizabeth Bennett, in contrast, shows very little interest in climbing their society’s hierarchy in this way, and even rejects Mr Darcy on his first proposal.

William Thackeray’s Vanity Fair, meanwhile, famously plays with our expectations of what to expect in a protagonist by placing the ruthlessly ambitious (and possibly murderous) Becky Sharp at the very centre of the novel, while her more amiable (but bland) friend Amelia is a secondary character. It was, in Thackeray’s own words, “a novel without a hero”, but in evolutionary terms Becky’s comeuppance, as she is ultimately rejected by the society around her, still signals a stark warning to people who might be tempted to put themselves before others. 

Bonnets and bonobos

Evolutionary theory can also shed light on the staples of romantic fiction, including the heroines’ preferences for stable ‘dad’ figures (like Mr Darcy in Pride and Prejudice or Edward Ferrars in Sense and Sensibility) or flighty ‘cads’ (such as the dastardly womanisers Mr Wickham or Willoughby). The ‘dads’ might be the better choice for the long-term security and protection of your children, but according to an evolutionary theory known as the ‘sexy son hypothesis’, falling for an unfaithful cad can have his own advantages since they can pass on their good looks, cunning and charm to his own children, who may then also enjoy greater sexual success.

The result is a greater chance that your genes will be passed on to a greater number of grandchildren – even if your partner’s philandering brought you heartbreak along the way. It is for this reason that literature’s bad boys may still get our pulses racing, even if we know their wicked ways.

Writers like Austen are intuitive evolutionary psychologists with a “stunningly accurate” understanding of sexual dynamics

In these ways, writers like Austen are intuitive evolutionary psychologists with a “stunningly accurate” understanding of sexual dynamics that would pre-empt our recent theories, Kruger said. “I think that’s part of the key for these stories’ longevity. [It’s why] Jane Austen wrote these novels 200 years ago and there are still movies being made today.”

There are many more insights to be gained from these readings, including, for instance, a recent analysis of the truly evil figures in fantasy and horror stories – such as Harry Potter’s nemesis Lord Voldemort and Leatherface in The Texas Chainsaw Massacre. Common features include a grotesque appearance appears to be designed to trigger our evolved fear of contagion and disease, and given our innate tribalism, villains often carry signs that they are a member of an “out-group” – hence the reason that so many Hollywood baddies have foreign accents. Once again, the idea is that a brush with these evil beings ultimately reinforces our own sense of altruism and loyalty to the group.

The novelist Ian McEwan is one of the most celebrated literary voices to have embraced these evolutionary readings of literature, arguing that many common elements of plot can even be found in the machinations of our primate cousins. “If one reads accounts of the systematic nonintrusive observations of troops of bonobo,” he wrote in a book of essays on the subject, The Literary Animal, “one sees rehearsed all the major themes of the English 19th-Century novel: alliances made and broken, individuals rising while others fall, plots hatched, revenge, gratitude, injured pride, successful and unsuccessful courtship, bereavement and mourning.”

McEwan argues we should celebrate these evolved tendencies as the very source of fiction’s power to cross the continents and the centuries. “It would not be possible to enjoy literature from a time remote from our own, or from a culture that was profoundly different from our own, unless we shared some common emotional ground, some deep reservoir of assumptions, with the writer,” he added.

By drawing on that deep reservoir, a story like the Epic of Gilgamesh is still as fresh if it had been written yesterday, and its timeless messages of loyal friendship remain a lesson to us all, 4,000 years after its author first put stylus to tablet.

David Robson is a freelance writer based in London. He is @d_a_robson on Twitter.

BBC Culture’s Stories that shaped the world series looks at epic poems, plays and novels from around the globe that have influenced history and changed mindsets. A poll of writers and critics, 100 stories that shaped the world, will be announced in May.

If you would like to comment on this story or anything else you have seen on BBC Culture, head over to our Facebook page or message us on Twitter.

And if you liked this story, sign up for the weekly bbc.com features newsletter, called “If You Only Read 6 Things This Week”. A handpicked selection of stories from BBC Future, Culture, Capital and Travel, delivered to your inbox every Friday.