PredictWise Blog

Overall Calibration Pretty Good ...

Bookmark and Share

In a preliminary look at Election Eve predictions for the 507 elections, we did pretty well.

Senate (34 of 35): I am going to hold out Louisiana for now and assume there were 35 elections. In those 35 election we had the binary winner in 34 of them, with NC going Republican, despite 85% for the Democratic incumbent. The average probability for the leading candidate was 95%, which translates into an expected outcome of 33, so we overshot by 1.

Governor (32 of 36): I am going to assume that the results in Alaska, Colorado, Connecticut, and Vermont all hold. In all of these elections we were leaning towards the current leader. Florida, Illinois, Kansas, and Maryland all leaned Democratic prior to the election and were captured by Republican. We had an average probability of 92% for the leading candidate going into the election, which translates into an expected outcome of 33, so we undershot by 1.

The overall average of the 71 elections was 93%, translating into an expected outcome of 66 correct elections. Which means we were relatively well calibrated at 66. In short, we won one more senatorial election than we should and loss one more gubernatorial election than we should.

House (419 of 435): There are still several house election in dispute, but assuming the current leaders prevail, our house election were 419 of 435. The average probability for the leading party was 96% which translates into an expected correctness of 417 seats. So, we are relatively well calibrated at 419 seats. Only two of the sixteen “misses” went Democratic versus Republican.

All “misses” in the senate, governor, and house elections, except for two house races, went Republican rather than Democratic. So the Republicans over-performed the predictions overall.

Election Night 2014

Bookmark and Share

1:30 AM ET: This is what under-performance versus the polls looks like:

10:33 PM ET:

10:13 PM ET: We have the senate at 95% for the Republicans, but that almost feels generous to the Democrats.

8:07 PM ET: The Democrats continue to over-perform the trandtional polling in the exit polling (with the exception of VA, which is tight). And, they are all but certain to capture both of the key must wins states of NH and NC. Georgia is tighter than expected as well. This is going to to be long night, but definitly better for the Dems than expected. 

7:34 PM ET: We just upgraded NC to 95% in light of the strong exit polls for the incumbent Hagan. VA is too tight to call right now, which is bad new for the Democrats. But, both GA elections are too close to call as well, which is bad news for the Republicans. Overall, the Democrats are slightly over-performing on the exit polls relative to the traditional polling. But, it is too early to say if the results are going to be biased in either direction this election cycle. Grab a snack, becuase the roller coaster at 8 PM ET!

6:36 PM ET: Kentucky has closed a part of the state and the early returns have GOP incumbent McConnell well ahead of 2008 in almost every location. He went into tonight at 100% and will win

3:20 PM ET:

1:30 PM ET:

1:15 PM ET: The table below will update all night tonight and all commentary by me will be in this spot. So, please check back for live updates starting just before 7 PM ET.

Link to House page ...

Note: Prob is "probability of victory" and EV is "expected vote share"

Congress - Likelihood of Party Control - After 2014 Election

2014 Senate - Likelihood of Victory for Democratic Candidate - Election Night

2014 Governor - Likelihood of Victory for Democratic Candidate - Election Night

Microsoft Prediction Lab's final probabilities

Bookmark and Share

Five weeks ago I launched a new website, with a few friends, including Miro Dudik and David Pennock, called Microsoft Prediction Lab. The website consolidates research into both non-representative polling and prediction games. I have spent years understanding how various raw data: polling, prediction markets, and social media and online data, can be transformed into indicators of present interest and sentiment, as well as predictions, of varying populations. Then, how decision makers allocate resources with the low latency and quantifiable market intelligence that we produce. Microsoft Prediction Lab allows us to continuously innovate not only on the path of raw data to analytics to consumption, but the collection of the data itself. Please take a look at this post on the mission of Microsoft Prediction Lab.

First, we would like the thank the thousands of active users who made this first game such an interesting and meanignful experience!

Microsoft Prediction Lab has a market for all 507 elections in the midterms: 36 senatorial, 36 gubernatorial, and 435 house. In each of these markets users can buy and sell contracts on the possible outcomes of each election. For example, in New Hampshire there are two possible outcomes: Democratic candidate Jeanne Shaheen and Republican candidate Scott Brown. A prediction on Shaheen would return 112 points for every 100 wagered, while a prediction on Brown would return 467 point for the same 100 points! If someone thought that Brown as undervalued, that there was a good return in wagering 100 points for 467, s/he should predict Brown and if s/he thought Shaheen was undervalued they should buy Shaheen. As people predict Brown, the return goes down and vice-versa.

The return that an investment settles on in a market is extremely correlated with the probability of the outcome. We show the translation of the price to the probability on the market alongside the return on prediction.

Further, the markets moved quite a bit over the last few weeks. Actually, there was movement in most of the 507 markets. In this market 85 people placed predictions, while others saw well into triple digits. To test how efficient that movement was, whether the crowd was supplying information, we captured the probabilities in all 507 races at about midnight on Election Eve.

These probabilities represent the probability of victory for the party just before 12:00 AM on Election Day. We look forward to checking back later this week to see how Microsoft Prediction Lab did ... and, for those of you playing the game, you have until 9 PM ET to keep the predictions coming!

Microsoft Prediction Lab - Final Probabilities

Democrats poised for (possible) great night … in governors elections

Bookmark and Share

The Democrats have a decent chance at having a great night in the governor’s races. There are 10 elections that I am following closely:

2 Tough, but possible pickups for the Democrats:

Wisconsin: this is a potentially a huge pick-up for the Democrats against a likely 2016 contender for the Republicans. Scott Walker is a conservative Republican leader in a solid blue state.

Maine: Paul LePage is a second unapologetic conservative Republican in a solid blue state. He is also in a tight spot.

3 Likely pickups for the Democrats:

Florida: neither the Democratic former governor Crist nor the current Republican governor Scott are pulling ahead in this nail-biter.

Alaska: independent Bill Walker (running with a Democratic lieutenant governor) is looking very strong against the Republican incumbent.

Kansas: Paul Davis is pulling ahead of conservative Republican Sam Brownback in this incredibly red state.

3 Likely holds for the Democrats:

Colorado, Illinois, and Connecticut: all three of these have looked tight, but are likely to remain blue governors in blue states.

1 Likely hold for the Republicans:

Michigan: this is a generally blue state, but there is a Republican incumbent heading towards a likely hold.

1 Likely loss for the Democrats:

Massachusetts: the only likely bad mark on the election for the Democrats is Martha Coakley. She is poised to lose against a moderate Republican Charlie Baker. Although, as a moderate Republican in a state with a long tradition of moderate Republican governors*, this is not a major ideological shift. *Note: I am referring to Governor Romney, a moderate, who should not be confused with Republican nominee for present Romney, who is not a moderate!)

Republicans at about 85% to capture U.S. Senate

Bookmark and Share

The Republicans are about 85% likely to take control of the U.S. Senate in January, 2015. This is going to happen, because the Democrats are going to win Blue States and the Republicans are going to win Red States. And, the Republicans are likely going to win two crucial Purple States (Iowa and Colorado). This election is not wave or a disaster for either party, but pretty much as should be expected. The most likely outcome is going to be the Democrats controlling 47-8 seats to the Republican 52-3.

The Democratic path to victory is very simple; they need to capture both New Hampshire and North Carolina, which are likely, and then three additional states of the five in play. The runoff system makes it very unlikely they will win in Georgie and Louisiana. And, I do not think Kansas’ Orman is going to make himself the swing vote with a 49 Democratic senate. Why would he do that if he can switch back in 2017 with seniority when the Democrats recapture the senate?

There is a possibility of a systematic polling bias against the Democrats (i.e., the Democrats will over-perform polls on Election Day). Nate Silver seems to think I am wrong (although the post is very long and does not address me directly, so it is hard to say for sure). He believes that the bias in 2012 is an historical anomaly. His argument is that states tend to break towards their fundamentals (i.e., Democratic states polls are biased against Democrats and Republican states polls are biased against Republicans). The problem is that he runs a regression with data from 1998 to 2012 and includes presidential polling along with senatorial. First, in 1997 the response rate for random digit dialing was still 36%, versus <9% in 2012. While I applaud the use of added observations, including time before the technical issues appeared for pollsters is a mistake that will smooth all treatment effects. Second, presidential polling is going to swamp senatorial polling. Not only in size, but in stability and accuracy. Presidential elections have more stable voter turnout.

Knowing there was an issue in 2012, I have no doubt that the pollsters will try to correct for any bias, but I believe it is likely that there will still be some bias against Democrats. First, with shifts in both coverage and non-response error moving fast it is not easy to correct for what happened in two or four years ago. Baseline data used for corrections is already obsolete (i.e., reliable lists of cell-phone only versus landline/cell phone are not updated fast enough!) and simply correcting for the error of four years ago is also obsolete (i.e., any correlations from four years ago are already wrong). This group of pollsters may under or over correct, but it is hard to see how they would be systematic unbiased. Further, they are technically very conservative people, so under-correcting is just more likely. Second, many pollsters are not knowledgeable enough to do that and will not bother. This group of pollsters are likely to favor Republicans.

Note: I had a very interesting early morning phone call today with Sam Wang of Princeton, so I expect he may touch on some of the same topics about bias today as well.

Any potential systematic polling bias, that favors the Democrats, will only get them Colorado and possibly Iowa; putting them at 49 seats. Colorado is most likely, because, with high population movement and Hispanic population, it is susceptible to bias. High population movement, young people with cell phones, leads to coverage error that favors Republicans. Hispanic population lead to non-response error, as Democratic Hispanics are less likely to answer English polls as Republican Hispanics. Iowa is the next most likely, as a Democratic leaning purple state shares some of the same attributes. It is less likely that Alaska or Georgia are going to be heavily hit by bias, as both states are much less susceptible. Kansas does not have a Democratic challenger!