Disclaimer: this post contains an affiliate link to the book Superforecasting: The Art and Science of Prediction
You might think that visiting a fortune-teller is just a bit of fun or it’s a massive waste of time and money. Yet predicting the future can be a very serious business for Intelligence Agencies. If you can forecast what other nations will do, you can be ready in advance. So who do you think is better at predicting future events? Intelligence agents with access to top-secret intelligence or a group of people who have no experience of espionage and have only the same access to the news as the rest of us? Welcome to the strange world of superforecasting.
Experts and chimps playing darts
Philip Tetlock is interested in predictions, or more accurately, he is interested in the predictors, how accurate they are and how they make their forecasts. In his 2005 book, ‘Expert Political Judgment: How Good Is It? How Can We Know?,’ Tetlock reflected that experts are little better at predicting than a dart-throwing chimpanzee. To put in less colourful language, experts only slightly outscore pure chance. Not that this is necessarily a criticism. Forecasting future events is very, very difficult.
In 2011, the US Intelligence Advanced research Projects Activity (IARPA) decided to find the best way to improve the forecasting ability of intelligence officers. To do so it organized a competition. Five research teams competed, including a team called the Good Judgement Project, which was co-created by Tetlock along with Barbara Mellers and Don Moore. Each team had to give daily forecasts to questions posed by IARPA. The Good Judgement Project performed so well that by the second year IARPA dropped the other teams.
The power of superforecasting
The Good Judgement Project showed us that there are people who are a lot better than dart-throwing chimps. Tetlock calls these people Superforecasters. The best of these can drastically outperform intelligence agents who have access to classified information. The superforecasters use little more than Google and their own smarts.
Another finding is that many heads are better than one. By combining the predictions of superforecasters, we can avail of the wisdom of crowds to make the predictions more accurate still.
“Superforecasting does require minimum levels of intelligence, numeracy, and knowledge of the world, but anyone who reads serious books about psychological research probably has those prerequisites,” Philip Tetlock, Superforecasting: The Art and Science of Prediction
So are some people really good at making lucky guesses? No, the psychologists believe that forecasting is a skill that can be trained and honed. Not everyone can be a superforecaster, but the people who may be good at predicting the future can train themselves to be better at it. One trait of superforecasters is the ability to put aside their own opinions while judging the probability of a future event. To be really good at predicting the future you only needs the basics of being smart, well-informed and having an open mind. The rest, seemingly, can be taught.
So, will the intelligence agencies of the world be overthrown by a collection of amateurs with wifi? Personally, I think the chances of that are pretty low. However, they may just get a lot better at predicting future events thanks to Tetlock’s research.