Analysis: How the bookies beat the pollsters in General Election predictions
Opinion pollsters face another post-mortem over their failure to predict last week's British election result, while world investors are paying heed to yet another win for betting markets.
Prime Minister David Cameron's Conservatives won an outright parliamentary majority in a major upset that came after months of polls, including nearly a dozen on election day itself, showed them neck-and-neck with the main opposition Labour Party.
Yet bookies and so-called 'prediction markets' consistently forecast the Conservatives would be the largest party and that Cameron would stay in office.
It was the latest in a long list of instances where the aggregated best guesses of punters putting money at stake has proved superior to studiously representative and modelled polling of people's voting intentions.
Professor Leighton Vaughan Williams is director of the Political Forecasting Unit at Nottingham Business School. He says bookmakers and betting markets have consistently been more accurate on major political events over the past 15 years, from US presidential contests to the Scottish independence referendum last year and March's Israeli election.
While polls and bookies often point in the same direction, Vaughan Williams claims the latter has been closer to the actual result in all 10 national votes he has studied since 2000.
"The power of the betting markets to assimilate the collective knowledge and wisdom of those willing to back their judgement with money has only increased in recent years as the volume of money wagered has risen dramatically," he said.
MARKETS LIKE MARKETS
For investment managers schooled in the art of measurable data rather than political or behavioural analysis, this sort of observable accuracy appears like gold dust – particularly when it refutes rather than reinforces the underlying polling.
Benjamin Netanyahu's shock win in Israel, for example, came after final opinion polls showed his Likud party trailing by two to four percentage points. Betting markets had put the chances of his return as prime minister at between 55 and 80 per cent.
When Scotland voted on seceding from the United Kingdom last September, polls showed support for independence rising sharply in the final few weeks of the campaign, with financial markets wobbling after one spiked above 50 per cent. But betting odds and prediction markets never tilted to 'Yes' at any stage.
"Even if they don't always beat the polls, they give you a more balanced opinion and I find betting markets very useful," said Valentijn van Nieuwenhuijzen, head of multi-asset investing at NN Investment Partners.
He said a key attraction of watching odds at online bookmakers such as Betfair, or percentage probabilities on prediction markets such as Hypermind, where punters trade the likely outcome of a political event as if it were a stock price, is that often emotional bias is stripped out.
That is because betting can be done from anywhere in the world and – at least in theory – is focussed on picking the correct answer rather than the outcome the punter would prefer.
Defenders of polling say direct comparisons with betting markets are unfair.
Opinion surveys are a snapshot of publicly stated intentions – honest or not – which when pooled into 'polls of polls' should get close to the result. And they do in many elections, the 2010 UK election being an obvious counterpoint to last week.
A considered gambler will use such polls alongside other information sources to refine his or her view, and therefore it should be no surprise that results of aggregated betting are often more accurate.
"Like all aggregators and forecasters, bookmakers and spreadbetters are in a privileged position to be able to use a whole host of data to predict outcomes," said Joe Twyman, head of political and social research at YouGov.
Other question marks include the sheer variety of betting methodologies and models and even the phrasing of questions.
The PredictWise website, for example, uses a variety of sources as a base for its forecasts including polls, prediction markets and bookies' odds.
For investors, NN's van Nieuwenhuijzen says, finding a market on the precise question you want answered can be tricky sometimes and it's difficult to systematically model betting odds for investment planning with so many varieties.
But for those seeking a winner, PredictWise – citing Betfair prices – estimates there's more than a 70 per cent chance Greece will still be in the euro by the end of 2015.
Betfair odds suggest Chuka Umunna will be the next leader of Britain's Labour party.
And Hypermind shows 50 per cent expect Spain's anti-austerity movement Podemos to emerge as only the third-biggest party at elections this year.