Features

How did the polls get it so wrong? Jim Messina knows

Cameron's data guru speaks exclusively to The Spectator

16 May 2015

9:00 AM

16 May 2015

9:00 AM

Not all the pollsters got it wrong. On the morning of the election, a set of strikingly accurate predictions was slapped on David Cameron’s desk. They had been compiled by Jim Messina and Lynton Crosby, the strategists who had been running a campaign derided as dull and repetitive. But, as their research showed, it was also effective. Messina is now back in his office in Washington DC.

‘We predicted 312 seats that morning to Lynton,’ he says. This was in line with the exit poll (316 Tory seats) that shocked Westminster. Yet, every day of the campaign, the polls had the Tories and Labour neck and neck. Did he ever doubt his methods? ‘No, not at all,’ he says. Crosby carried out daily tracker polls and Messina was conducting analytics. ‘Both of us had the same data, so we were very sure.’

Their working assumption was that all the polls used by the newspapers — YouGov, ComRes, Ipsos MORI — were talking rot. ‘Having been through the Obama campaign,’ says Messina, his assumption was that ‘public polling is wrong’. So what were the mistakes? ‘One, a lot of them were using a 2010 view of the electorate. Two, they weren’t naming the candidates in the seats. I assume that is a recipe to get it absolutely wrong — because some incumbents do really well.’

The British pollsters generally asked which party people supported; not about which individual they’d choose as an MP. Failure to make that distinction, Messina believed, led the pollsters up a blind alley.


Also, he and Crosby were running far larger samples, with a mixture of online and phone calls. The public pollsters used online or telephone research, but never both. As he put it, only the Tories had that ‘kind of mix — there certainly wasn’t anyone doing modelling of the electorate’.

And British pollsters need to work out why their figures were so wrong for so long: ‘It was so clear, especially for the entire last month, that they were wrong. But none of them wanted to hear about it.’

The Liberal Democrats won just eight seats, but even on election day YouGov predicted that they would win 23. ‘You really have to not be paying attention to get that,’ says Messina. Most Lib Dem losses fell to the Tories. Was that a product of his data, or Crosby’s strategy?

‘Lynton was definitely the inspiration behind that strategy and believed very, very deeply in it and really understood it,’ he replies. ‘At his request, we built a model of the seats of everyone we felt could switch between us and the Lib Dems. We had our field team contact those people directly and that clearly worked.’

Labour also hired its own ex-Obama campaign adviser, David Axelrod, who offered advice on messaging instead of polling and data.

Messina seems unimpressed. ‘To this day, I can’t tell you what Labour’s message was, other than, “We don’t like the Tories.” But until the famous Ed rock or Ed stone, you sort of had no idea what they were running on. When you are trying to do that five days before [an election], you’re in deep, deep trouble.’

Crosby’s research, he says, showed ‘that Cameron had taken tough steps, things were starting to get better — and Miliband wasn’t offering anything new. That combination made it very, very difficult for them to win.’

You might disagree with half of it, but you’ll enjoy reading all of it. Try your first 10 weeks for just $10


Show comments
  • StoryHugh

    I have my own theory as to why the private Party Pollsters have a better picture of what is going on than the national pollsters with public results.

    In every election the parties have tellers standing on the door of every polling station. They ask for your polling card number. This means that they have a very good picture of who actually votes in your constituency, even in boring local elections. When they take an opinion sample they are contacting actual voters.

    National polling companies have a random sample of people of whom perhaps 35% will not actually vote. These people are obliging and want to help, but many of them probably convey their impression of the campaign’s momentum and the big picture, rather than expressing the opinion of a single committed voter.

    The snap polls after every TV “debate” or leadership grilling showed that Cameron had won. These proved an accurate predictor of what was going to happen because viewers of these programmes are people who are interested in politics and likely to vote.

    The BBC unhelpfully reported the debates as either a victory for Miliband or having no clear winner. This impression of a tight race was picked up by the less politically interested population and fed into the random samples by the national pollsters.

    This is my armchair theory. Would be interested to know what the pros think.

    • davidofkent

      There were no Party Tellers at my polling station. In any case, it was a bit late for the public pollsters by then wasn’t it?

      • StoryHugh

        The parties have a better idea about who voted at the previous elections. If somebody bothered to turn out for a council election, you can be pretty sure that they are going to vote in the general election. This is a reality check that they can run against their opinion polls taken at any time.

        • Yorkieeye

          You are right Hugh. That is why we buy the marked register after polling day. We compare who turned up with our canvas data ( assuming people are telling us the truth when we canvas them for voting intention.

          • StoryHugh

            Presumably that means national pollsters buy the marked register too ? And know who has a record of voting.

            If so, my theory isn’t quite so good. But I do think that Political Parties have a massive army of unpaid volunteers collecting information. They have a lot of data about individuals’ whole history of voting. This can be surely be used by the expert pollsters that they hire in to get a more accurate picture about who is telling the truth about voting intentions, including whether they are likely to turn up.

          • Callipygian

            In America we talk a lot about ‘likely voters’. We don’t pin too many hopes on ‘UNlikely voters’, i.e. those that are not following the campaign at all, don’t know the main candidates, and have rarely or never voted in the past.

          • Yvon & Barry Stuart-Hargreaves

            In America you barely have anything like democracy.

          • Callipygian

            Yet more highly informed and insightful commentary from Yvon & Barry.

          • Yvon & Barry Stuart-Hargreaves

            Why, thank you very much.

          • Dogsnob

            Nor does the UK.

          • Yvon & Barry Stuart-Hargreaves

            What amazes me is that outfits like YouGov even survive. Their crappy little finger in the breeze polls are less than useless. One hopes no major company pays good money for the pointless tripe about consumer preferences that these chartlatans spew out.

    • John Moss

      Tellers are there to collect elector numbers only. They are barred by law from asking how people voted.

      The number go back to a campaign centre to be knocked off the list of voters, then “knock-up” lists are produced for local activists to go and root out their supporters. A crude for and against tally may be kept, or if running software, a more detailed estimate based on “voting intention” data from canvassing can be seen.

      This all happens on polling day so has no bearing whatsoever on prior polls.

      • StoryHugh

        But surely knowing who turns up and votes at elections is some of the most valuable information of all. You don’t have to know how they voted, just that they have a record of bothering to vote. Then when you conduct an opinion poll you can look at a group of people who are known voters. This is a very valuable reality check.

      • StoryHugh

        Or if they don’t use this information, they are missing a trick.

  • kathee

    I remember lying in my room when I was in high school and writing in a journal to my future husband. I’d write all sortss of notes and questions and things I’d wonder or ask this man when I eventually met him. I would wonder where he was and what he was doing and if he was thinking about me too. It has always been such a strong desire in my heart to find a wonderful man to marry, someone who would love me and cherish me and appreciate me for the person I am. I always thought I would get married right out of college, just like my parents, so when that plan didn’t work out, I started to get discouraged. A school mate snatched my future husband away from my arms just because she had spiritual powers, all hope was lost to me before i came across the help doctor (prayerstosaverelationship@gmail.com
    ) who i confided in, i told him my long story and he helped me regain back my lover with his prayers which is now my husband today. if you have any problem email the help doctor (prayerstosaverelationship@gmail.com
    ).

    • blandings

      I have a similar tale to tell, though i found salvation in a couple of martinis and a hot chick.
      (cheaper than your help doctor I suspect)

  • Malcolm McCandless

    The public polls were predicting a Labour minority government. Internal polling was predicting a minority Tory government. The exit poll predicted a minority Tory government. The election result was a majority Tory government.

    It would seem to me everyone got it wrong.

    • WFB56

      Yes, except the voters in the actual poll.

    • Yvon & Barry Stuart-Hargreaves

      Certianly the 10 million who voted Tory got it wrong, selfish and easily led.

      • pedestrianblogger

        Selfish? “I have never understood why it is “greed” to want to keep the money you have earned but not greed to want to take somebody else’s money.” ― Thomas Sowell, Barbarians inside the Gates and Other Controversial Essays

        You want to take “somebody else’s money” and represent YOUR greed as altruism.

        • Yvon & Barry Stuart-Hargreaves

          You want to contribute less,but use our roads, our surgeons, our police and our well educated workforce.

          • pedestrianblogger

            No, I don’t.

  • ant

    The ‘Tories are Evil’ line did for Labour. Rolling out the likes of Steve Coogan, who demonstrably has no political sensibilities beyond hating ‘the tories’, merely re-enforced the total lack of a coherent message. That translated into the polling in the ‘shy Tory’ phenomenon that Labour interpreted as gains for them, while those of us with more of a nose for such things saw it for what it was and bet accordingly. The total mystification of the ‘virtuous celebrity’ at Labour’s catastrophe has been a joy to behold. Maybe Coogan should stick to what he used to be good at. Coke and hookers.

    • nekomuna celo

      Pollsters say main factor was lazy labourites not bothering to vote, rather than shy tories

  • kathee

    I remember lying in my room when I was in high school and writing in a journal to my future husband. I’d write all sorts of notes and questions and things I’d wonder or ask this man when I eventually met him. I would wonder where he was and what he was doing and if he was thinking about me too. It has always been such a strong desire in my heart to find a wonderful man to marry, someone who would love me and cherish me and appreciate me for the person I am. I always thought I would get married right out of college, just like my parents, so when that plan didn’t work out, I started to get discouraged. A school mate snatched my future husband away from my arms just because she had spiritual powers, all hope was lost to me before i came across the help doctor (prayerstosaverelationship@gmail.com
    ) who i confided in, i told him my long story and he helped me regain back my lover with his prayers which is now my husband today. if you have any problem email the help doctor (prayerstosaverelationship@gmail.com
    ).

  • Gerschwin

    People lie. That simple.

    • MA0

      “People lie.”

      Almost. They don’t have time to lie consciously, and why pump a party you don’t support? I believe that people answer pollsters with their public selves, but when in the privacy of the booth they do what their private conscience dictates. Only a gross taboo can elicit this personal dichotomy, and that taboo is racism. People didn’t vote for Miliband because he does not look British enough. Only that is shameful enough to cause voters to lie to themselves in conversation with pollsters. Polling will improve when people become less ashamed of their rational and irrational racism. The misleading polls and the UKIP vote both result from the same issue. A significant number of Celtic/Roman/Saxon/Norman British people won’t publicly admit, even to a pollster, that they would prefer to be governed by a Celtic/Roman/Saxon/Norman British Prime Minister.

      • Callipygian

        What an absolute slander on the British people! Not only that, you’re wrong.

        • MA0

          There goes your public self.

          I don’t see it as a slander. People all over the world are racist. The British may be some of the less racist, but why pretend we are 100% race blind. It is this kind of refusal to talk honestly about race that is storing up a world of trouble. In my opinion, it would be perfectly reasonable for the British electorate to have a preference for non-immigrant prime-ministerial candidates. In most other countries that would be considered normal, not slander. The USA even has a law against immigrant presidents.

          You are an anti-white racist.

    • Yvon & Barry Stuart-Hargreaves

      We are becoming like Greece and Russia. Nations of liars who do not even trust close family.

      • Gerschwin

        Don’t worry YBSH, now the Tories are properly in truth will always triumph.

  • Yvon & Barry Stuart-Hargreaves

    There is a large element of the “too ashamed to admit it Tory”,nobody likes to admit they are slefish and easily led. But the pollsters ask the wrong people. YouGov polls are pointless and manipulated. They only ask the 68% of people with a Broadband connection. Phone polls only ask those with a landline. Randon street polling asks those out shopping in the daytime.
    Fact is in all polls 85% of people,say they intend to vote. The pollsters predicted a 74% turnout, the reality was 66%.

    • balance_and_reason

      Oh dear …..snide carping after your dog lost….

      • Yvon & Barry Stuart-Hargreaves

        My dog was never going to win.We just won Brighton.

        • balance_and_reason

          watermelons.

    • MA0

      It’s not shame, it’s avoidance of abuse. We’re just tired of brain-washed lefties calling us ‘nasty’, ‘selfish’, ‘fascist’, ‘racist’, ‘sexist’, ‘homophobe’, et cetera just for not believing in your obsolete Marxist dystopia. I vote against your brain-washed leftism because I actually care about the harm that is done to the ‘poor’ (i.e. the less well-off) by envy politics. I want the less well-off to have jobs, not handouts of other people’s money. Luckily not everyone is as stupid as you.

      • Yvon & Barry Stuart-Hargreaves

        Touch Tory.

  • Civil War in UKIP

    So not prompting for the MP’s name was one of the reasons the polls were wrong due to local loyalty…. despite the fact that almost all the deposed Lib Dems were very-well-known and often very liked both locally and nationally…. and the Tory challengers were by definition mostly unknown?

    Mr Messina is making it up afterwards. He’s as much in the dark as the pollsters were.

    • John C.

      I’m sure you’re right, it wasn’t a matter of local personality. If they had to be told the names of the candidates, then they clearly didn’t have a favourite in mind.
      No, it’s really a simple combination:
      1) voters didn’t feel like gambling away the at least steady economy on a horse with a shaky record.
      2) the violent anti-Tory rhetoric from Scotland told them that Labour would be manipulated by the SNP.
      The strong suggestion that only Left-leaning people could show compassion and sympathy encouraged shy Tories not to declare their hands in public. Who would admit to being Nasty? But they’d made up their minds thanks to 1) and 2) some time ago.

      • Morsefan

        I think you are right about all of your reasons for why the Tories did so well, but they do not explain why the public polls didn’t pick up on those dynamics. Jim Messina is also right that public polls in the UK tell you very little you need to know to take poll information and predict an outcome: no second preferences, no party identification, etc. But he was not trying to predict a result; he was trying to create one. He won’t tell the secret of his success, but even though name recognition is not “everything” in the UK as it is in the US, it may be useful to know. Maybe it turns out to be closely associated with something he can’t easily poll for directly. He may have a formula that tells him that if someone does not recognize the names of any candidates, that person is not a likely voter or that at the margins if someone knows the name of the candidate, the chance that person will vote for that candidate’s party is much higher. (In the U.S., people rarely vote for people at the top of a ticket that they have not heard of.) In the UK, maybe you will vote for someone you have never heard of if you really dislike the one you have heard of. Also, he’s polling over and over, so he’s getting information on how public opinion is shifting and why. Anyway, he can then even use that information to create the result he wants: for example, he can know who the party should make sure gets taken to the polls and how to use available resources to do so. And he can have near certainty of how that particular person will vote and factor that into his model. No public pollster can do that.

    • Morsefan

      You assume this, but one might argue the election returns said otherwise. I think Mr. Messina is making a different point: public polling in the UK is pretty unsophisticated. You just aren’t getting useful information to tell you what is happening. He may simply not be willing to be specific about everything that made a difference to him, but the way you ask a question matters. I was shocked by it when I realized there seemed to be almost no information about individual candidates. I cannot believe public polls in the UK just hold the important information back, but I don’t know that I ever saw data on second preferences or how people planned to vote based on party identification. Surely polls in a parliamentary system are meaningless without those. I am not sure if the polls were tracking or not, and sometimes when the data was collected was not clear. Even if “polls are bad” you can often see indicators of movement, shy voters, etc., if you look at additional information. Of course everyone was shocked. Once something was off, everything was.

  • Rupert

    This really shows what a disaster Nick Clegg was for his party as due to his unpopularity the Tories were able to target and win a lot of the lib dem seats.

    • Yvon & Barry Stuart-Hargreaves

      Nick bloody Clegg and his shower of MP’s and failures are wholly responsible for delivering Ten years of Tory cuts and mean minded pettiness.

      • Rupert

        No Brown is responsible. Clegg went to brown to offer him a coalition and Brown told him to get lost .

  • balance_and_reason

    where is Telemachus now that Labour has been whipped?…we need to laugh and poke fun at the fool.

    • Dogsnob

      He is busy attacking, attacking, attacking.
      Or perhaps playing Candy Crush Saga.

  • evad666

    Do any polling companies poll further North than Hampstead?

  • Mordwinoff

    Polling! A waste of time. Just talk to a London Cabbie and you will know who is going to win.

    • MA0

      This is true. All the cabbies I chatted to before this election were for Tory and UKIP. Fifteen years ago many were for Blair.

    • Yvon & Barry Stuart-Hargreaves

      Talk to Uber drivers they are cheaper.

  • Hippograd

    Messina works for Obama in the US and for Cameron in the UK. How peculiar, given that Obama’s and Cameron’s politics are so utterly different.

    http://en.wikipedia.org/wiki/Jim_Messina_%28political_staffer%29

    I suspect members of the stale pale male community are at work.

  • WFB56

    David Axelrod appears to have been a complete waste of money. Lucky for the Tories.

  • The Shambolic Skeptic

    It’s always the marginals and the swing voters in the marginals that make a difference. Identify them and poll them and you’ll get a solid answer. Party pollsters figured this out long ago.

    These national “which party are your voting for?” polls are worse than worthless.

  • trace9

    I got it right, when I called Ed’s Battle-bus his Death-Rattle bus the day before polling, in a comment hereabouts. & Cheaper too – it’s called ‘common sense’, aaand some awareness of the world you live in. Also right when I’d previously commented that the new 5 year-per–Poll legislation was a gross mistake – one’s heard Dave’s thinking of deleting it now..

    What profits it a Wise Man
    To live in a World of Fools
    More than to escape into Himself..
    Bugger All.

  • samuelafugglas

    The commie journalists on BBC tried to make up bogus results in advance to impress and change the mind of the few!

  • Harryagain

    The polls were deliberately manipulated.
    Why?
    If they had simply been wrong, the results they came up with would have been random.
    As it was, they were all “wrong” the same way which can’t be coincidence.
    So it was deliberate misinformation but who did it benefit?
    Answer the Tories, their enemies thought they could relax.

Close