A Google Translation – useful but not accurate
English Bulgarian Czech Danish Dutch French German Greek Italian Spanish Swedish
FacebookTwitterGoogle BookmarksLinkedinRSS Feed

Opinion polls: How could Gallup predict that Mitt Romney would win the presidential election in US?

On November 6, 2012 CNN announced that Barack Obama would win the US Presidential election because it was clear after the election results from the state of Ohio that he was going to be in office for another 4 years.

This outcome was a big surprise for the opinion poll institute Gallup, as late as in October Gallup had predicted a victory for Mitt Romney and announced it for instance on Fox News. Mitt Romney was, according to the opinion poll, leading by 7 percent-points:  Romney 52 %   Obama 45 %. This prediction continued until the last poll just before the election, but by this time the lead had been reduced to only 1 %.

This didn’t shake Obama’s chief adviser David Axelrod – he was willing to bet on the fact that Obama would win. With all due respect to the polls, he said, he relied more on sure pieces of information. His staff actually stood forward and criticized Gallup’s polls for being “way out of line”. But Gallup didn´t flinch. The editor-in-chief appeared on television and defended the methods used at their institute: “Our methodology is extremely solid, and we are very open about how we do it, so I will say that we are doing great, and it is not unusual, of cause, that people would fight back at you – they don´t like what you are finding”.

In the Republican Party the delight was great. The former adviser to George W. Bush, Carl Rove, said he could almost now guarantee a Romney victory. This was the first time that Romney had hit the 50 % in a Gallup poll, and the President had never done that. No candidate who had led in mid-October with 50% or more in the polls had ever gone on to lose”. But history is one matter, reality another.

A couple of months after President Obama’s re-election, Gallup announced that they would make a thorough evaluation of their measuring methods. How could Gallup be so wrong? In collaboration with researchers from the University of Michigan they started testing and analysing their results. On the June 3, 2013 the results came in the form of a seventeen page report containing explanations on why it went wrong.

The first point was a matter of the way Gallup chose the respondents who the institute called (by telephone) to ask who they are going to vote for. Gallup had chosen their survey respondents in an unsystematic way by phoning randomly generated telephone numbers. But that method was rather expensive because the employees at Gallup ended up calling numbers that didn’t exist or called firms instead of voters. Instead they choose to call random numbers from the telephone book. This made the whole process cheaper. But the problem was that not all people are listed in the phone book, and according to the evaluation report the group of respondents was heavily weighted towards elderly white persons, who generally voted Republican. This was one of the main explanations why the results they obtained were to Romney’s advantage.  Gallup was aware of this possibility, but they thought they had compensated for this unrepresentative sample by also calling people with mobile phones.

There was an interesting historical point from the Presidential election of 1936 where Franklin D. Roosevelt defeated the republican Alf Landon. A number of previous polls from the respected Literary Digest Magazine had predicted the correct result of five earlier presidential elections. In 1936 they carried out a gigantic opinion poll with 10 million respondents. The results of this poll suggested there would be a landslide victory for Alf Landon by 12 %. But in the end that didn’t happen. The problem with this poll was that it was made by randomly selecting telephone numbers and car registrations. However, during the Depression years in the thirties, it was primarily the rich, typically Republicans, who were the owners of phones and cars. The consequences were that the number of Republican voters were over-represented in the surveys. At that time, a young man had predicted that this survey had gone seriously wrong, he was a certain George Gallup. This was the start of The Gallup Institute who did the surveys in 2013.

The second explanation for the Gallup poll inaccuracy was the impact of time zones. The Gallup Institute didn’t take them into account when making phone calls. This led to differences in the social-economic status of the respondents as Gallup employees were making calls at a certain time, but people in one time zone were having their meals and therefore refused to participate, while people in other time zones were preparing the dinner. Given these structural differences within the time zones this method resulted in an over-representation of respondents from one time zone and of one part of the region.

The third reason was probably the most important. It was about the way that Gallup makes sure that people being asked were going to vote. In the USA this factor is very important because, at the previous election, the turnout was only just more than 50 % of the eligible population. The way Gallup secures this, was by asking people a lot of questions such as how often they voted, how much they had paid attention to the election campaign, did they know where to go to vote. In this way Gallup was able to identify whether the people being called were more likely to vote. But this method pointed the poll results towards Mitt Romney because Obama’s voters were often not very interested in the campaign and they had not voted in previous elections. For this reason they were classified by Gallup as probable non-voters. They were filtered out from the research results and this suggested Mitt Romney had an advantage.

Read more:

MorningJoe: http://www.nbcnews.com/id/3036789/ns/msnbc-morning_joe/vp/52092027#52092027

Gallup, The editor- in-chief Frank Newport is answering the question: What went wrong in 2012? (4th of June 2013).

New York Times: http://thecaucus.blogs.nytimes.com/2013/06/04/gallup-identifies-factors-that-led-to-its-g-o-p-skew-in-november/?_r=0


Source:  A Danish award winning radio/TV-program called´”Detektor” (June 2013).  The program “detects” and analysis outspoken numbers from private and public organisations and from the media and  Government.

/Marianne Hjortlund 11/2013.