Cast your mind back across the turbulent events of recent history. Did you foresee President Obama’s election before he was even elected as a Democratic candidate – or did you back Hillary Clinton? How about the Arab Spring – could you hear the revolution in the first tremors of dissatisfaction? And did you faithfully predict the recent Ukraine crisis?
If you answer yes to these questions, you could be a “super-forecaster”, someone who is able to foresee the outcome of world events with astonishing accuracy. This has nothing to do with the reading of tea leaves; nor do you have to be a seasoned political pundit. Some super-forecasters are just everyday people, from all walks of life, who happen to have astonishingly perceptive judgements.
Over the last few years, psychologists have found a series of “hidden” talents for extraordinary mental skills that can go unnoticed by the people harbouring them. There are “super-recognisers”, for instance, who can easily remember people’s faces, even if they had only met the person once, years before. Other gifted individuals include super-tasters, who have an intense perception of flavour and super memorisers who naturally recall events from almost every day of their lives.
Those talents might be largely down to our genes, whereas forecasting ability should, in theory, be down to experience and learned skill. Yet top political experts tend to perform very poorly at crystal ball gazing: Philip Tetlock at the University of Pennsylvania has found they do only slightly better than chance. “Chimps randomly throwing darts at the possible outcomes would have done almost as well as the experts,” is how one political scientist summarised the findings to the New York Times.
Needless to say, military intelligence agencies have been watching closely. Inspired, in part, by one of Tetlock’s reports, the US Intelligence Advanced Research Projects Activity (IARPA) sponsored the four-year-long “Good Judgement Project” that aimed to find new approaches to political forecasting. Taking the form of a tournament, it recruited thousands of participants from all walks of life to test their forecasting skills on a range of questions, such as “Will Robert Mugabe cease to be President of Zimbabwe by 30 September 2011?” or “Will Greece remain a member of the EU through 1 June 2012?” Rather than simple yes or no answers – which cannot reflect the inherent uncertainty of life – they were asked to estimate the probability that each event would occur.
Three years into the tournament, Tetlock’s team has now released some of its first results to the journal Psychological Science, and at the annual meeting of the Association for Psychological Science in San Francisco last month.
One of the main aims was to see whether certain forecasters were consistently good. So after one year Tetlock checked the results of more than 2,000 participants and siphoned off the top 2% – the so called “super forecasters”, who then formed teams for the rest of the tournament. At the end of the second year, their performance was up to four times more accurate than other participants.
Although political expertise may have helped, they came from a variety of professions. “One of the best forecasters was a pharmacist,” says Tetlock.
As you might expect, these elite forecasters tended to score better on measures of intelligence than the other participants. But they all shared one other trait too: open-mindedness. In everyday life, open-mindedness may be mistaken for having liberal political views, but in psychology it is thought to reflect how well you deal with uncertainty. Crucially, open-minded people tend to be able to see problems from all sides, which seems to help forecasters overcome their preconceptions in the light of new evidence. “You need to change your mind fast, and often,” says Tetlock.
Another trait of effective forecasting that Tetlock highlights is self-awareness – understanding your own foibles. Even expert forecasters may fall into certain traps, so Tetlock’s team decided to test whether an hours’ simple training could help his forecasters avoid the most common mistakes. He is reluctant to give too many details of the methods, in case it influences his forecasters’ future performance, but he points out that too often forecasters begin by taking an “inside view” of a problem; when considering whether Mugabe would remain in power, for instance, they might begin by looking for measures of unrest in the country.
Yet research suggests that you could come to more accurate predictions if you instead take a step back and simply look at past historical data. So in the case of Mugabe, you might consider how long the average dictator clings to power, before refining your estimate. Other strategies were aimed at reducing known cognitive biases. For instance, research has shown people tend to make better decisions if they are reminded of common pitfalls, such as the tendency to exaggerate the risk of particularly frightening events, like a terrorist attack; they could also remember to consider both the best and worst case scenarios of a situation, since that opens the mind to the full range of possibilities and helps to question your basic assumptions about the event. These tactics may seem obvious, but all the available evidence on human irrationality suggests they are easily forgotten – even by those who should know better.
Tetlock is also looking into ways to exploit the forecasters’ collective intelligence. Many psychologists believe that experts working in teams perform worse than those working individually, since they soon become “yes men” who reinforce each other’s biases. “It becomes the madness of the crowds,” says Tetlock. But with some training on how to critique others’ views, and respond positively to alternative viewpoints, the participants achieved the best results when they were allowed to cooperate more actively.
Ultimately, the team hope these discoveries could change the way governments make decisions. “It’s not about the business of prophecy – it’s the business of clarity,” he says. “We want to increase the intellectual honesty of predictions.”
The ongoing tournament could also help to highlight ways that all of us can boost our forecasting skills in everyday life. Need to make a major decision about your future? Embrace uncertainty and identify your biases. Of course, if you’re a super-forecaster already, you probably saw that advice coming.