Predictions

The Peculiar Blindness of Experts

Unfortunately, the world’s most prominent specialists are rarely held accountable for their predictions, so we continue to rely on them even when their track records make clear that we should not. One study compiled a decade of annual dollar-to-euro exchange-rate predictions made by 22 international banks: Barclays, Citigroup, JPMorgan Chase, and others. Each year, every bank predicted the end-of-year exchange rate. The banks missed every single change of direction in the exchange rate. In six of the 10 years, the true exchange rate fell outside the entire range of all 22 bank forecasts.

In 2005, tetlock published his results, and they caught the attention of the Intelligence Advanced Research Projects Activity, or IARPA, a government organization that supports research on the U.S. intelligence community’s most difficult challenges. In 2011, IARPA launched a four-year prediction tournament in which five researcher-led teams competed. Each team could recruit, train, and experiment however it saw fit. Predictions were due at 9 a.m. every day. The questions were hard: Will a European Union member withdraw by a target date? Will the Nikkei close above 9,500?