DavidMRothschild's blog

Microsoft Prediction Lab: 2014 Recap

Bookmark and Share

Microsoft Prediction Lab tested the wisdom of the crowd in 507 elections this fall and did pretty well (here are the posted final predictions from election eve): 33 of 35 (so far) in the U.S. Senate, 30 of 36 in the gubernatorial elections, and 419 of 435 in the U.S. House. This is in terms of binary outcomes (i.e., who won and loss), but I will get into the probabilities below.

In the senate, there were two reasonable and well calibrated “misses”. The final prediction was 61% that Greg Orman would knock off incumbent Republican Pat Roberts in the Kansas. And, 62% that incumbent Democratic senator Kay Hagan would hold off Thom Tillis in North Carolina.

2014 election effect on 2016 election

Bookmark and Share

There are three main effects of the 2014 election the 2016 election. First, the Republicans are slightly more likely than before the election to capture the presidency, but the Democrats are still favored. Second, Scott Walker is much more likely to get the Republican nomination, while Jeb Bush is slightly more likely. Third, Mitt Romney is much less likely to get the Republican nomination. There is not really any effect on the Democratic nomination.

The Democratic nominee is 58% likely to win the 2016 presidential election; this is down ever so slightly from before 2014 Election Day. Presidential elections have a much larger voting pool, which is more Democratic, than midterm elections. And, I will let other people debate the motivation of the votes on Election Day 2014, but Obama will not be on the ballot in 2016.

Scott Walker shot up as the major solid, right-wing Republican during the 2014 elections. He won reelection convincingly in a Democratic state, Wisconsin. But, the key thing, is that unlike Mitt Romney or other blue state Republicans, he ran as a solid right-wing Republican.

A closer look at that GOP over-performance

Bookmark and Share

There are 35 senate elections (excluding Louisiana) and 36 gubernatorial elections. We had expected vote shares for all of them, in terms of two-party vote share, which were primarily generated from traditional polling.

In 54 of 71 (76%) elections the Republican candidate over-performed (28 of 35, 80%, senate and 26 of 36, 72%, governor). The average error (the bias in one direction) was 2 percentage points (the average absolute error was just 2.8 percentage points). The bias was a little more extreme in the gubernatorial elections (2.3 percentage points), than in the senatorial (1.8 percentage points). Since almost all errors were in the same direction, the absolute error is not much larger than the error.

Overall Calibration Pretty Good ...

Bookmark and Share

In a preliminary look at Election Eve predictions for the 507 elections, we did pretty well.

Senate (34 of 35): I am going to hold out Louisiana for now and assume there were 35 elections. In those 35 election we had the binary winner in 34 of them, with NC going Republican, despite 85% for the Democratic incumbent. The average probability for the leading candidate was 95%, which translates into an expected outcome of 33, so we overshot by 1.

Governor (32 of 36): I am going to assume that the results in Alaska, Colorado, Connecticut, and Vermont all hold. In all of these elections we were leaning towards the current leader. Florida, Illinois, Kansas, and Maryland all leaned Democratic prior to the election and were captured by Republican. We had an average probability of 92% for the leading candidate going into the election, which translates into an expected outcome of 33, so we undershot by 1.

Election Night 2014

Bookmark and Share

1:30 AM ET: This is what under-performance versus the polls looks like:

Congress - Likelihood of Party Control - After 2014 Election

2014 Senate - Likelihood of Victory for Democratic Candidate - Election Night

2014 Governor - Likelihood of Victory for Democratic Candidate - Election Night

Microsoft Prediction Lab's final probabilities

Bookmark and Share

Five weeks ago I launched a new website, with a few friends, including Miro Dudik and David Pennock, called Microsoft Prediction Lab. The website consolidates research into both non-representative polling and prediction games. I have spent years understanding how various raw data: polling, prediction markets, and social media and online data, can be transformed into indicators of present interest and sentiment, as well as predictions, of varying populations. Then, how decision makers allocate resources with the low latency and quantifiable market intelligence that we produce. Microsoft Prediction Lab allows us to continuously innovate not only on the path of raw data to analytics to consumption, but the collection of the data itself.

Microsoft Prediction Lab - Final Probabilities

Democrats poised for (possible) great night … in governors elections

Bookmark and Share

The Democrats have a decent chance at having a great night in the governor’s races. There are 10 elections that I am following closely:

Republicans at about 85% to capture U.S. Senate

Bookmark and Share

The Republicans are about 85% likely to take control of the U.S. Senate in January, 2015. This is going to happen, because the Democrats are going to win Blue States and the Republicans are going to win Red States. And, the Republicans are likely going to win two crucial Purple States (Iowa and Colorado). This election is not wave or a disaster for either party, but pretty much as should be expected. The most likely outcome is going to be the Democrats controlling 47-8 seats to the Republican 52-3.

The Democratic path to victory is very simple; they need to capture both New Hampshire and North Carolina, which are likely, and then three additional states of the five in play. The runoff system makes it very unlikely they will win in Georgie and Louisiana. And, I do not think Kansas’ Orman is going to make himself the swing vote with a 49 Democratic senate. Why would he do that if he can switch back in 2017 with seniority when the Democrats recapture the senate?

Hidden Errors and Overconfident Pollsters

Bookmark and Share

Written with Sharad Goel and Houshmand Shirani-Mehr

Election forecasts, whether on HuffingtonPost's Pollster, New York Times’ Upshot, FiveThirtyEight, or PredictWise, report a margin of error of typically 3 percentage points. That means that 95% of the time the election outcome should lie within that interval. We find, however, that the true error is actually much larger than that, and moreover, polls historically understate support for Democratic candidates.

Election Update - 11/2, 2 Days

Bookmark and Share

The Democrats are likely to lose the senate for two years. My predictions have been consistently more bullish on Republican victory than any of the other main forecasters: New York Times’ Upshot, FiveThirtyEight, HuffingtonPost’s Pollster, Princeton’s Sam Wang, etc. And, to be frank, the data is more generous to the Democrats than my gut, but I am obliged to run with the data.

The Democrats will have 47 seats if they take all of their certain races, along with New Hampshire and North Carolina. Of course, New Hampshire and North Carolina are not certain, but for the sake of this exercise, let us assume the Democrats take those seats. There are just eight other seats that are even remotely in play, and the Democrats would have to win three of them to get to a 50/50 tie, where Joe Biden is the tie-breaker.