Prof. publishes book on 'superforecasters'

By Joe Pires 11/03/15 11:38pm

Ordinary citizens may be able to out-predict professional analysts and pundits with confidential information, writes one Penn professor in his new book.

Management and Psychology professor Philip Tetlock, along with journalist Dan Gardner, assert that even without years of training and confidential information, one can still correctly predict world events in their newly-published book, "Superforecasting: The Art and Science of Prediction." 

The book is based on the results from the Good Judgement Project, a group of citizen forecasters started by Tetlock and his wife, Marketing and Psychology professor Barbara Mellers.

“Basically, the question that IARPA [Intelligence Advanced Research Projects Activity] wanted to answer was ‘what are the best possible ways to ask people to elicit probability forecasts’, and ‘what are the most accurate ways of combining those forecasts from a large group of people who are crowdsourced from all over the world?’” Mellers said.

The book's thesis is that political and economic forecasting, unlike the divinations common in modern-day pundits and analysts, could be treated like weather forecasts: very accurate in the short-run, but increasingly imprecise as time goes on. For Tetlock, a major difference between the forecasting produced by experts and by those within the GJP is the style of thinking.

“It goes back to an ancient poem by a warrior-poet Archilochus. The fox knows many things, but the hedgehog knows one big thing,” Tetlock said. “Hedgehogs tend to be theorists, animated by one big idea … But there’s a big idea that animates and organizes a lot of their thinking, whereas foxes are harder to classify, politically.”

In his book, Tetlock argues that the fox’s way of thinking is preferred over the hedgehog’s thinking in regard to forecasting. He compares modern forecasting to the history of medical research — people adopted the hedgehog’s way of thinking and tended to assume they were right about certain things, explaining away anything that contradicts their theories.

With programs like the GJP and IARPA forecasting competitions, hypothesizing future political and economic events can become more scientific — and therefore more accurate, Tetlock said.

Tetlock also posits that with the right training, anyone can become a forecaster, with rapid improvements coming from a short amount of training.

“It’s possible for everyone to learn to do it somewhat better if they follow certain basic training guidelines, and the basic training guidelines don’t take very long to master, so you can achieve improvements of 10 percent plus accuracy over the course of an entire year with a training module that lasts less than an hour,” Tetlock said.

Tetlock describes himself in the book as an “optimistic skeptic,” recognizing that the benefits and increasing popularity of forecasting will come slowly.

“I don’t think culture changes happen fast. I think it is a gradual process. But I think the stars are moving into alignment for some major changes in how we do things,” Tetlock said. “It’ll be painfully gradual. There will be many twists and turns, but I think that the advantages of running debates in this more civilized way will become apparent.”

For those interested in forecasting, Tetlock is teaching two classes — Psychology 275 and Management 276 — on the topic next semester.

Please note All comments are eligible for publication in The Daily Pennsylvanian.