Most Polls Are Snapshots, but the Xbox/YouGov Panel Shows Voters in Movies (Syndicated on the Huffington Post)
DavidMRothschild on October 22, 2012 @ 6:02PM
Confused by the incredible movement and variation in the daily stream of national popular vote polls? The Xbox/YouGov panel is your antidote to the noise; the panel paints a very clear picture of how the vote intentions of Xbox LIVE users have changed over the past three weeks. After the first presidential debate on Oct. 3 there was a record jump in respondents switching toward support for Romney and that rise in support persisted for a few days. The vice presidential debate stopped the bleeding for Obama and the second presidential debate on Oct. 16 gave Obama a few strong days. During this entire time the number of undecided voters has slowly drifted downward.
For each day between Sept. 24 and Oct. 21, we estimated the percentage change in support for each candidate. This estimate is based on people who were interviewed on a particular day and at least once in the preceding five days, so that they reflect the actual changes of individual voters, not aggregate changes in different samples of voters. The change is calculated by computing the percentage of persons giving each possible response (Obama, Romney, Undecided, Other) in their most recent interview and the percentage distribution of the same voters on the current day. The percentage change is the difference of these two percentages, weighted by the baseline proportion in each group. We have been polling Xbox LIVE users on their voter intention continuously since Sept. 22. There is an average of 6,502 paneled users reflected on a given day and a total of 40,988 at some point since September 24.
This chart is not nearly as dramatic as what recent national telephone polls have shown, with wide swings following the first presidential debate, despite there being relatively few undecided voters in most polls. For example, Obama was leading by eight points in Pew's September survey and trailing by four points in Pew's October survey conducted shortly after the first presidential debate. Similarly, Gallup's tracking poll from Sept. 27 through Oct. 3 had Obama leading by four points and it now has him trailing by one point among registered voters and six points among likely voters. Our daily tracking poll shows similarly dramatic changes.
Most polls, including the ones noted above, are based on new samples for each new poll. When you compare two polls, even from the same pollster, the difference depends on the samples being the same. If the composition of the samples change, the polls may give a misleading indication of how much movement there really is. Although these are labeled "probability samples" or "probability-based samples," the response rates are typically 10 percent or less, so most of the probability comes from the likelihood that a respondent chooses to participate, not the chance of selection.
By comparing the distribution of background variables in different polls, it is easy to see that the same people, or even the same type of people, are not responding repeatedly. For example, respondents to the September Pew poll reported voting 47 for Obama and 32 for McCain, while respondents to the October Pew poll voted 42 for Obama and 37 for McCain. Where did the 2008 Obama voters go?
No one would claim that our panel is a representative sample of the population -- the poll respondents are overwhelmingly male and a majority under 30 -- but, we have the advantage that (a) we interview a lot of people every day, and (b) the same people participate multiple times per week. So we have a large population of the same people, telling us how they plan to vote repeatedly. We can observe real change in this audience, rather than trying to infer change from different samples with unknown characteristics. And, with so few people switching support during the course of an election, it takes a massive sample to record the quantities we observe.
We should not be surprised that the number of McCain supporters surged in the October Pew poll relative to the September Pew poll; selection bias is a serious issue in polling. Imagine that you are an Obama supporter belonging to demographic group X and Pew randomly picks to call you a few days after Romney's 47 percent video came out. You would be totally stoked to answer and tell them how much you love Obama. Now imagine Pew randomly picks to call you a few days after Obama's disastrous first debate. This time, you avoid the phone call and the person that replaces you might share your demographics but is a Romney supporter that is more than happy to sing Romney's praises. In that scenario, a shift in the polls reflects enthusiasm and cooperation, as much or more than an actual shift of support.
You can learn a lot about what's going on with large, high frequency data collection from a self-selected and non-representative population. We understand that there is a lot more research to go, but even at the very start this new venture we are seeing meaningful and relevant trends in regard to the 2012 election.
If you have an Xbox, please participate in our daily polls on Xbox LIVE and do not forget to watch the debate tonight live on your Xbox and participate in our within debate polling!
This article is syndicated on the Huffington Post.
Romney could win the popular vote and lose the election (Originally posted on Yahoo!'s "The Signal" Blog)
DavidMRothschild on October 22, 2012 @ 2:01PM
To be clear, we don't necessarily think Romney will win the popular vote. Standing in national polls does not predict actual vote share, and those polls are swinging back in the president's favor. Romney's odds of winning a majority of the ballots, however, are higher than his odds of winning at least 270 electoral votes. This year, the Electoral College unfairly favors Obama: Romney must carry Florida, Virginia and Ohio, while Obama needs only one of them.
The latest Gallup tracking poll of likely national voters has Romney up by 7 points. A lot of virtual ink has been spilled on how and why Gallup's poll has become an outlier: I suggest Alan Abramowitz, Mark Blumenthal or Nate Silver on the subject. The more meaningful aggregations of polls at Pollster and RealClearPolitics both report a statistical tie between Obama and Romney in the national polls.
Of the three states Romney needs to win, Florida (80.0 percent likely for Romney) and Virginia (61.9 percent likely for Romney) are leaning in his favor. Ohio has stubbornly remained in Obama's camp, with a 65.0 percent chance of going to Democrats. Obama held firm in the state after his Oct. 3 debate debacle, Ohio's economy is doing relatively well, and early voting is already under way, minimizing the impact of late-breaking events.
Let the pundits tie themselves into knots over the latest Gallup poll. As usual, what America wants is immaterial compared to the desires of the Buckeye State.
DavidMRothschild on October 18, 2012 @ 12:35AM
Woody Allen is often quoted as saying that "80 percent of success is showing up." Hackneyed though this expression has become, it applies quite accurately to the election as it stands today. President Barack Obama showed up at the debate on Tuesday night and stabled his teetering campaign.
Given the wide consensus that Obama did not mentally show up for the first confrontation with former Gov. Mitt Romney, his combativeness and general vigor appeared to convince the television audience that he still has some fight left in him. Instant polls suggest that Obama scored well overall and, more critically, with undecided and leaning voters. Polls of overall voters are not that meaningful, because most people will say their candidate won. But surveys of undecided and leaning voters, like those from CBS and Xbox/YouGov, give us valuable clues. Obama clearly outperformed Romney in both.
Our real-time forecast, heavily influenced by prediction markets at this point in the campaign, ticked up nearly 3 percentage points during last night's debate.
DavidMRothschild on October 15, 2012 @ 6:39PM
Xbox, in conjunction with the polling company YouGov, has launched a new poll for this election cycle. The goal of the poll is threefold. First, Xbox wants to create an engaging product for its users. Second, the polling team wants to be able to provide meaningful insight into the 2012 election. Third, both Xbox and YouGov recognize the research angle of this work; this poll is the product of cutting-edge interactive television and polling techniques that, while not supplanting the established methods today, will be instrumental in both entertainment and information gathering in the future.
In the interest of full disclosure, as a member of Microsoft Research in New York City, I am the head pollster on the Xbox side of this team.
There are two connected polling operations: the daily poll and the live event polling. First, every day on Xbox Live's election channel there is a new poll that asks between three and five questions. New users answer a slate of standard background questions the first time they enter. Every day the poll asks the standard voter intention question and then groups the remaining questions into daily themes: economy, social issues, etc. Second, Xbox is streaming the pool feed for all four debates and asks question on the bottom of the screen periodically to their users during the debate. A few baseline questions at the beginning are followed by a stream of questions reacting to the segments. In both polling settings respondents self-select into the polls.
Above is an example of the live event polling on the Xbox Live from Thursday's VP debate. Click on the image to watch the video!
Respondents are enjoying the experience; they are using it in droves and coming back. First, over 20,000 unique respondents per day are answering the daily polls and there were over 30,000 unique respondents answering each question during Thursday's VP debate. Second, Xbox is getting very high repeat engagement with upwards of 20 percent of users taking the daily poll any given day having already taken a daily poll in the past. And, the engagement levels during the debates have been at least a magnitude larger than standard engagement for web-based polling.
After Thursday night's VP debate Xbox was able to release timely and meaningful data on the reactions from their viewers. The sheer quantity of the respondents answering each question provides us with a very meaningful number of undecided and leaning respondents. Biden dominated Ryan in all of our top-line questions; undecided registered voters thought he won the debate, was more truthful and more capable of being president. Similarly, respondents leaning toward Obama were more overwhelming for Biden in all three questions than respondents leaning toward Romney were for Ryan. We have no prior reason to believe that the undecided voters in our poll behave too differently from average undecided voters, but since this a self-selected non-representative sample, further research is ongoing. Yet, parsing the data by both age and gender, we saw perfectly consistent results.
Above is a snippet of results from Thursday night's Xbox Live/YouGov poll. Approximately 30,000 Xbox Live users responded to each question posed during the debate on 10/11/12. The survey was designed and analyzed by YouGov.
Expect a regular, meaningful stream of results from the Xbox Live/YouGov poll between now and Election Day. Harnessing both the large sample sizes and repeated respondents, we are busy learning about our repeated users from our unbalanced panel that have switched over the last few weeks. We are also exploring the value of randomly generated subsamples of daily respondents. While, we embrace the oversampling we have hard to get demographics, like 18-29 males and young families.
While I am happy to help create a fun and meaningful product for 2012, what really excites me about this poll is the research and potential of this technology. I am huge believer in the value of the probability based scientific polling and I use the aggregated totals from HuffPost's Pollster as a key ingredient in my real-time forecast of the 2012 election. But, my job is to engage in academic, peer-reviewed style research that pushes the boundaries of information aggregation.
One of those boundaries is to see how far we can increase the value of information supplied in polls of self-selecting and non-representative respondents. We hope that by gathering massive amounts of respondents and having repeated responses by many, we will be able to overcome some of the statistical biases inherent in our sample. We are experimenting with new weighting, new debiasing methods, new questions, and new interactive interfaces. We recognize the limits and embrace the advantages: quickly updating questions, massive coverage, and cost-effective respondents.
So many people today watch TV with a second screen around and we are giving them something fun, related, and meaningful to do with extra energy; we have enacted the promise that is interactive television with the hope that it will lead us towards the future of information aggregation.
If you have an Xbox, watch live during Tuesday's debate. Either way, I look forward to updating you on the results.
This article is syndicated on the Huffington Post.
The Senate is the Democrats’ to lose as five more states shift (Originally posted on Yahoo!'s "The Signal" Blog)
DavidMRothschild on October 15, 2012 @ 2:07PM
The Democrats currently control 53 of 100 seats in the Senate. Nearly half of those—23—are up for re-election this year, while the Republicans are defending only 10 seats. That fact, combined with an electorate none too pleased with incumbents, made for a grim picture for the majority party at the start of this cycle.
And yet, the Democrats now have a 75 percent likelihood of controlling the next Senate, possibly by a comfortable margin.
In early September, we saw four critical races shifted to the Democratic column: Missouri, Wisconsin, Massachusetts and Virginia. Today, the Democrat is up in all four races, though three of the four remain competitive. Now, we're seeing five other races once considered safe for the Republican showing signs of equivocation: Nevada, North Dakota, Indiana, Arizona and Montana.