EU referendum: Why the use of data by EU campaign groups should be taken with a tablespoon of salt
You really shouldn’t take too much stock in the figures that either side are shouting about in the build up to the EU referendum.
Indeed, both sides are putting forward arguments for why we should remain or leave the 28-member bloc ahead of the vote that’s expected this year.
But for Simon Tilford, deputy director at the Centre for European Reform, the numbers aren’t helpful at all: “The whole idea of putting concrete figures on this is nonsense. It is impossible to be anywhere near as precise as the In and Out camps are attempting to be on this question.”
That’s not to say there isn’t some useful analysis in some of the reports, but Jonathan Portes, research fellow at the National Institute of Economic and Social Research, says the numbers are highly speculative “and often much worse than that”.
Farcical is the word Anand Menon, director of the UK in a Changing Europe and professor at King’s College London, uses to explain the use of the data.
“There have been all sorts of dubious figures banded about”, he says. “In social science we talk about replicability, so if you make a claim you need to show the data so that someone else can get the same data, get the same sums, and come up with the same answer. There’s a massive absence of that in everything that is being said.”
Read more: In the EU debate we should be wary of apocalyptic predictions
Take a report published by the Confederation of British Industry that says membership brings a benefit of £3,000 for each family in the UK. Menon says that it’s impossible to work out where that number comes from: “The CBI say something like ‘based on respectable studies’.” And he reckons that’s not good enough, especially given that the data was reproduced in a leaflet by Britain Stronger In Europe.
Yet James McGrory, chief campaign spokesperson, disagrees: “In stark contrast to our opponents, all the arguments we use are backed-up by genuinely independent, expert research. Unlike some of those campaigning to leave, we believe voters deserve the facts, not spurious figures to support bogus arguments."
But that’s without even mentioning that the benefits and costs of EU membership aren’t distributed equally. “You’ve got to wonder if the benefit of EU membership for a banker in London, a bank that does all the euro-denominated trading, so earns a fortune from its ability to trade are the same as the benefits for a bloke in Stoke who’s unemployed – and the answer has got to be no,” says Menon.
Unsurprisingly, the Out campaign is sceptical of the In campaign’s data. Richard Tice of Leave.EU said: "Some of the figures being peddled in this referendum are extremely dubious. For example, Stronger In's headline claim that EU membership is worth £3,000 to every British household was dismissed by FactCheck as 'not based on any real evidence' all the way back in 2013, but they have repeated it over and over again to try and pound it into the collective unconsciousness without the media making much effort to hold them to account."
But the Out campaign is guilty of this too. A recent Vote Leave paper said that the costs of Britain’s membership to the EU had hit £500bn. But they’d simply “added up what the UK paid into the EU budget since 1973. The problem with that is twofold: it overlooks the rebate completely, and negates any other potential benefits that the EU brings,” says Menon.
He’s joined by Portes, who said: “On the Leave side the "we send £20 billion a year to Brussels" ignores the rebate which is absurd in this context.”
Though perhaps the most famous example is that three million jobs would be put at risk if the UK leaves the EU. True enough, “it is reasonable to say that, very approximately, there are 3m jobs linked to EU trade,” says Portes. “However, and more importantly, it is not remotely sensible to use this, explicitly or implicitly, as a measure of the number of jobs that are in any sense threatened by Brexit.”
Why? Because whether jobs are threatened, let alone lost, would depend on what trading arrangements were negotiated in a post-Brexit world. Portes thinks that we have “no sensible way of knowing at present” if jobs would be put at risk. It's not likely that all trade would disappear if Britain left the EU and if the same companies keep trading with Europe, the same jobs will remain.
Read more: Is Labour hurting itself by campaigning to stay in the EU?
These precise figures often boil down to the assumptions that have been made in the calculations. “Going for a precise number is farcical – it’s ridiculous, it makes no sense, and the assumptions you’re making are so huge as to make the exercise pointless, essentially,” Menon says.
To get a real idea of what’s going on, one has to look at what information campaigns actually base their assumptions on because if one assumes, as say the out camp does, that EU membership is a constraint on British trade, then leaving the EU, by that reasoning, will boost British trade. So, as Tilford states: “One has to scrutinise their assumption that EU membership is a constraint on trade.”
It’s not all that different to the political campaigns in the build up to the General Election, Tilford says. “We see that if one looks at the manifestos of the General Election what both major parties in the UK argue and how they site evidence is heavily influenced by their own assumptions and own interests.”
“The campaign on this is no different from any other political campaign.”
Ultimately, though, the problem is that both campaigns’ use of data lacks credibility.
Of course it's reasonable for Britain Stronger In Europe to argue that the economics strongly favours staying in, and that households would lose if we left – and for the other side to argue the opposite – but to say "leaving will cost you X" isn't very credible, says Portes.
Or take the question of what arrangement we would come to with the EU after voting to leave. Menon says the “In side will say that Norway implements a massive proportion of EU law, but go to Daniel Hannan’s website and he’ll be saying it’s like 10 per cent. The question is how the hell do we know what’s what?”
Indeed, how can both sides make claims about the costs or benefits of EU membership that directly contradict one another?
Because ultimately the campaign “isn’t about facts, it’s about mood”, says Menon. “The campaign is more specifically about risk. And what both camps are doing is desperately trying to make the alternative to what they want to sound riskier than what they want. It’s all about who can project fear more effectively.”
And that’s worrying because, as Tilford points out, the situation now is one where both sides are “in a battle of unsubstantiated claims. It now means people discount the economic argument. So people are going to be swayed by other arguments such as immigration, nationalism and sovereignty”.