PredictWise Blog

Golden Globes Predictions: Boyhood and Birdman edition

Bookmark and Share

The Golden Globes are Sunday, January 11 and, as usual, we have some bold, market-based, predictions on PredictWise. The methodology for the Golden Globes is the exact same as the Oscars, which has not been previously tested out-of-sample for the Golden Globes, but we are excited to see how they do. We have predictions for 10 categories and here are our pre-event predictions:

We have the drama Boyhood looking very strong in Best Director, Best Picture (drama), and Patricia Arquette as Best Supporting Actress. We have the comedy Birdman rivaling Boyhood with a favored position in Best Picture (comedy or musical), Michael Keaton as Best Actor (comedy or musical), and Best Screenplay. Eddie Remayne leads the Best Actor (drama) category from The Theory of Everything, Julianne Moore from Sill Alice as the Best Actress (drama), Emily Blunt from Into the Woods as Best Actress (comedy or musical), and J.K. Simmons from Whiplash as Best Supporting Actor. The closest categories are Best Actress (comedy or musical) with Amy Adams from Big Eyes in a close second, and Best Screenplay with The Brand Budapest Hotel and Boyhood both challenging.

We expect to get 8 out of 10 correct with an average probability for the leading nominee at 82%. Enjoy the show and follow the live predictions here:

Microsoft Prediction Lab: 2014 Recap

Bookmark and Share

Microsoft Prediction Lab tested the wisdom of the crowd in 507 elections this fall and did pretty well (here are the posted final predictions from election eve): 33 of 35 (so far) in the U.S. Senate, 30 of 36 in the gubernatorial elections, and 419 of 435 in the U.S. House. This is in terms of binary outcomes (i.e., who won and loss), but I will get into the probabilities below.

In the senate, there were two reasonable and well calibrated “misses”. The final prediction was 61% that Greg Orman would knock off incumbent Republican Pat Roberts in the Kansas. And, 62% that incumbent Democratic senator Kay Hagan would hold off Thom Tillis in North Carolina.

In the gubernatorial elections, there were six “misses”. The final predictions had challenger, Democrat, Paul Davis overtaking incumbent Sam Brownback in Kansas with 69%. Democratic challengers, Charlie Crist (78%) and Mike Michaud (55%) both failed in their attempts in Florida over Rick Scott and in Maine over Paul LePage, respectively. Everyone missed Democrat Anthony Brown (77%) to beat Lawrence Hogan for the open seat in Maryland. Finally, Democrat Dannel Malloy was 49% to hold off Thomas Foley in Connecticut and Republican Sean Parnell was 78% to hold off Bill Walker in Alaska. These misses are consistent with the top poll-based forecasters, like Nate Silver.

The forecasts for the senatorial and gubernatorial elections were well calibrated. The average probability for the favorite in the senatorial elections was 90%; we expected 32 of 35 to be correct and got 33 of 35. In the gubernatorial election we had an average probability of 83%; we expected 30 of 36 to be correct and got 30 of 36.

With so much information available, we were not surprised to see the prediction games mirror the major poll-based forecasters, that is why the U.S. House predictions are the most exciting; 419 of 435 elections is a very strong track record! It is a little stronger than the expected 409, but we are not complaining. Of the 16 misses, 4 went Democratic and 12 went Republican. This is not so surprising in a year of Republican victories.

2014 election effect on 2016 election

Bookmark and Share

There are three main effects of the 2014 election the 2016 election. First, the Republicans are slightly more likely than before the election to capture the presidency, but the Democrats are still favored. Second, Scott Walker is much more likely to get the Republican nomination, while Jeb Bush is slightly more likely. Third, Mitt Romney is much less likely to get the Republican nomination. There is not really any effect on the Democratic nomination.

The Democratic nominee is 58% likely to win the 2016 presidential election; this is down ever so slightly from before 2014 Election Day. Presidential elections have a much larger voting pool, which is more Democratic, than midterm elections. And, I will let other people debate the motivation of the votes on Election Day 2014, but Obama will not be on the ballot in 2016.

Scott Walker shot up as the major solid, right-wing Republican during the 2014 elections. He won reelection convincingly in a Democratic state, Wisconsin. But, the key thing, is that unlike Mitt Romney or other blue state Republicans, he ran as a solid right-wing Republican.

Jeb Bush, not on the ballot, had a good day as the moderate Republican standard-bearer; that means that Mitt Romney lost the day. Another moderate Republican, Chris Christie, should be happy about Republican governors having a good day, but Jeb Bush and Scott Walker offset any of his joy.

Much more about the 2016 election in the coming months and years.

A closer look at that GOP over-performance

Bookmark and Share

There are 35 senate elections (excluding Louisiana) and 36 gubernatorial elections. We had expected vote shares for all of them, in terms of two-party vote share, which were primarily generated from traditional polling.

In 54 of 71 (76%) elections the Republican candidate over-performed (28 of 35, 80%, senate and 26 of 36, 72%, governor). The average error (the bias in one direction) was 2 percentage points (the average absolute error was just 2.8 percentage points). The bias was a little more extreme in the gubernatorial elections (2.3 percentage points), than in the senatorial (1.8 percentage points). Since almost all errors were in the same direction, the absolute error is not much larger than the error.

PredictWise went into Election Day with 27 elections that had a non-negligible probability for both candidates. Of those elections 20 or 74% moved more Republican; the bias was 1.8 percentage points and the average absolute error was 2.2 percentage points. Much of this was driven by Maryland where the error was 9.9 percentage points! The gubernatorial elections had a bias of 1.6 percentage points (1 percentage point without Maryland) and the senatorial elections had a bias of 1.9 percentage points.

The interesting thing is breaking the group up into likely Republican and likely Democratic. In 13 of 14 elections that leaned Republican, the Republican did better than expected. In the remaining 13 elections the Republicans did better than expected in 7 of them. Actually, without Maryland, the Democratic leaning states had no bias, and tiny average absolute error of 1.5 percentage points.

What does this all mean for traditional polling and voting? That is the puzzle that we will explore over the next few weeks and months.

Overall Calibration Pretty Good ...

Bookmark and Share

In a preliminary look at Election Eve predictions for the 507 elections, we did pretty well.

Senate (34 of 35): I am going to hold out Louisiana for now and assume there were 35 elections. In those 35 election we had the binary winner in 34 of them, with NC going Republican, despite 85% for the Democratic incumbent. The average probability for the leading candidate was 95%, which translates into an expected outcome of 33, so we overshot by 1.

Governor (32 of 36): I am going to assume that the results in Alaska, Colorado, Connecticut, and Vermont all hold. In all of these elections we were leaning towards the current leader. Florida, Illinois, Kansas, and Maryland all leaned Democratic prior to the election and were captured by Republican. We had an average probability of 92% for the leading candidate going into the election, which translates into an expected outcome of 33, so we undershot by 1.

The overall average of the 71 elections was 93%, translating into an expected outcome of 66 correct elections. Which means we were relatively well calibrated at 66. In short, we won one more senatorial election than we should and loss one more gubernatorial election than we should.

House (419 of 435): There are still several house election in dispute, but assuming the current leaders prevail, our house election were 419 of 435. The average probability for the leading party was 96% which translates into an expected correctness of 417 seats. So, we are relatively well calibrated at 419 seats. Only two of the sixteen “misses” went Democratic versus Republican.

All “misses” in the senate, governor, and house elections, except for two house races, went Republican rather than Democratic. So the Republicans over-performed the predictions overall.