Wednesday, May 16, 2007

Political polls and markets in the 2004 Iowa Caucus

Matthew Yglesias has a good post on the predictive power of polling:
To make a long story short, national horserace polling data right up to the day of the Iowa Caucus had no informational value whatsoever about the outcome of the nomination race.
I've been looking at the Iowa Electronic Markets a lot recently, and was curious if they were any more accurate than the polls at predicting the winner of the Iowa Caucus in 2004, and they were by a slight margin. Markets performed as poorly at predicting who would take Iowa in the weeks and months before the caucus, but they effectively captured Dean's steady decline beginning in early December, long before we saw this decline reflected in the polls or the media. Additionally, the market trends were far more accurate than the polls at predicting the likely winner--Kerry--in the week before the caucus. It's not to say markets were absolutely accurate--Dean's shares were still higher than Kerry's on Jan. 18--but they were more accurate than the poll data.
In the six days before the caucus, Kerry's share price surged on the IEM; the polls, by contrast, showed some fluctuation and a mild uptick over the course of the week, but his Jan. 12 poll numbers were almost identical to those on Jan. 18, the eve of the caucus. At the same time, Dean's share price fell significantly between Jan. 12 and Jan. 18, nearly converging with Kerry's shares just before caucus day. Dean's poll numbers declined slightly during that week before, but did not hint at how close the race had become on the homestretch. (See page 12 of this report for the market graph.)
The conventional wisdom seems to be that Dean collapsed in the last week before the caucus, but the market shows a fairly steady decline from his all-time high in early December. As the authors of the report point out, his shares declined .25 from a high of .76 on Dec. 9 to .51 on Jan. 18, and only dropped another .16 on Jan. 19. A .16 decline on caucusing day points to a good deal of residual uncertainty about his ability to win, but the .25 decline clearly indicates that many had already left the party long before the 19th.
Oddly enough, the polls in Yglesias's graph show the opposite trend: in early December, Dean's poll numbers had yet to peak, and his Jan. 18 numbers were higher than his early December numbers. Kerry's poll numbers, meanwhile, declined between early December and the caucus's eve. It's true that the market did not predict Kerry's rise as effectively as it predicted Dean's return to earth, but usefully, it still demonstrated more predictive power than the polls.
Perhaps it goes without saying, but these differences in predictive power shouldn't really surprise us: polls and political markets are asking different questions, using significantly different methods. In short, polls ask random individuals who they intend to vote for, while markets ask self-selected traders who they think will win.
A couple trail-off questions: Don’t polls still tell us something about where the campaigns stand today even if they tell us little about events of the distant future? At the very least, whatever this something is seems to hold a lot of weight inside campaigns. Is poll data more useful to a campaign than the confusing aggregation of data in a share price?

1 comment:

Ben said...

I know this is not exactly relevant, but for anyone interested in politics, this full transcript of the Comey testimony is a must-read. Among other things, it reveals that Arlen Specter is a funny guy.