https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2660628

Distilling the Wisdom of Crowds: Prediction Markets versus Prediction Polls

Management Science, Early View, Forthcoming

37 Pages Posted: 15 Sep 2015 Last revised: 28 Nov 2017
Pavel D. Atanasov

Pytho LLC
Phillip Rescober

Independent
Eric Stone

Independent
Samuel A. Swift

University of California, Berkeley - Institute of Business and
Economic Research (IBER); Tepper School of Business
Emile Servan-Schreiber

LUMENOGIC
Philip Tetlock

University of Pennsylvania
Lyle Ungar

University of Pennsylvania
Barbara Mellers

University of California, Berkeley - Marketing Group; University of
Pennsylvania - Marketing Department

Date Written: March 1, 2017
Abstract

We report the results of the first large-scale, long-term,
experimental test between two crowd sourcing methods – prediction
markets and prediction polls. More than 2,400 participants made
forecasts on 261 events over two seasons of a geopolitical prediction
tournament. Some forecasters traded in a continuous double auction
market and were ranked based on earnings. Others submitted probability
judgments, independently or in teams, and were ranked based on Brier
scores. In both seasons of the tournament, last day prices from the
prediction market were more accurate than the simple mean of forecasts
from prediction polls. However, team prediction polls outperformed
prediction markets when poll forecasts were aggregated with algorithms
using temporal decay, performance weighting and recalibration. The
biggest advantage of prediction polls occurred at the start of
long-duration questions. Prediction polls with proper scoring,
algorithmic aggregation and teaming offer an attractive alternative to
prediction markets for distilling the wisdom of crowds.

Keywords: Prediction, Forecasting, Prediction Markets, Crowdsourcing

JEL Classification: C82, C93, D84

Suggested Citation:
Atanasov, Pavel D. and Rescober, Phillip and Stone, Eric and Swift,
Samuel A. and Servan-Schreiber, Emile and Tetlock, Philip and Ungar,
Lyle and Mellers, Barbara, Distilling the Wisdom of Crowds: Prediction
Markets versus Prediction Polls (March 1, 2017). Management Science,
Early View, Forthcoming, Available at SSRN:
https://ssrn.com/abstract=2660628