The Chairman of the Joint Intelligence Committee has written to the Cabinet Office, saying that it was ‘highly likely’ that the Assad regime was responsible for that attack. The Labour party says that is not enough for war. It has called instead for ‘compelling evidence’ that the Syrian regime was responsible. Which raise the obvious question: what (apart from politics) is the difference between ‘highly likely’ and ‘compelling evidence’?
The Joint Intelligence Committee’s letter assessing the evidence is as follows (these are extracts):
‘We have assessed previously that the Syrian regime used lethal CW [chemical weapons] on 14 occasions from 2012. This judgement was made with the highest possible level of certainty following an exhaustive review by the Joint Intelligence Organisation of intelligence reports plus diplomatic and open sources. We think that there have been other attacks although we do not have the same degree of confidence in the evidence. A clear pattern of regime use has therefore been established.’
On claims that the 21st August attack might have been faked, or carried out by opposition forces, they say:
‘We have tested this assertion using a wide range of intelligence and open sources, and invited HMG and outside experts to help us establish whether such a thing is possible. There is no credible intelligence or other evidence to substantiate the claims or the possession of CW by the opposition. The JIC has therefore concluded that there are no plausible alternative scenarios to regime responsibility. We also have a limited but growing body of intelligence which supports the judgement that the regime was responsible for the attacks… Some of this intelligence is highly sensitive but you have had access to it all. Against that background, the JIC concluded that it is highly likely that the regime was responsible for the CW attacks on 21 August.’
It is highly likely that the [Assad] regime was responsible… In the end there is no 100 percent certainty about who is responsible… – David Cameron, to House of Commons, 30th August.
So on some things the JIC has ‘the highest possible’ level of certainty; on some it thinks they are true but ‘does not have the same degree of confidence’; and on others again it has a ‘limited but growing body of intelligence.’ They assess their net conclusion as ‘highly likely.’
This is perfectly proper. Any decision, on any subject, has to be based on whatever evidence can be obtained, and on the degree of certainty – the likelihood or otherwise – that the evidence is factual. This last bit is the realm of statistics.
Mathematically, likelihood (or probability if you prefer) is measured on a scale of 0 to 1, with 0 being impossible, 1 being certain, and 0.5 being equally likely either way. No-one has ever defined ‘highly likely’, (or, in this context, ‘compelling’) but a number of, say, 0.8 would mean that you would be right 8 times out of ten, and wrong the remaining two. In horse-race betting jargon, that would be odds of 4 to 1 on. Your horse would be a clear favourite, but by no means certain to win. If the same race were to be run five times, identically, with the same horses, and the same jockeys, on the same track in the same conditions, your horse would win four of the races and lose the other one.
[We need] the production of compelling evidence that the Syrian regime was responsible for the use of these weapons. – Ed Miliband, Labour party leader, in amendment to House of Commons motion.
Except of course that it doesn’t work that way. You can toss coins lots of times identically; and you can spin roulette wheels lots of times identically; but you can’t re-run horseraces that way: the circumstances each time will be different. And you can’t re-run chemical weaponry attacks that way either. Real life is a one-off. Either Assad was responsible for the chemical attacks or he wasn’t. Which is why some statisticians (known as Frequentists) say that attaching likelihood numbers to such things is absurd.
But bookmakers make a very good living by attaching likelihood numbers – or probabilities, or odds – to what they (and the punters) believe about horse-races (and elections, and other one-offs of real life.) The statistics they use are called Bayesian statistics, and the statisticians they use are called Bayesian statisticians. Frequentist statisticians say that Bayesian statistics are based on nothing but subjective opinion, and so are completely meaningless; there was a time when Frequentist statisticians would not even be seen in the same room as Bayesian ones. But the embarrassing thing about Bayesian statistics is that in real life it works – even though (and this embarrasses even some Bayesians) it is based on nothing more that subjective opinion (and a rather brilliant mathematical equation.)
So possibly a Bayesian statistician might care to put a numerical value on ‘highly likely’ or on ‘compelling’ in this case. It would be based on their subjective opinion of the evidence, just as the JIC’s assessment was. It would probably not have made a lot of difference to the outcome of the debate. Which way each politician voted – for war or against war – depended more on that politician’s gut feeling about when a war is justified and when it is not than on anything else.
We cannot ignore the lessons of the calamitous Iraq war. We need safeguards to ensure that all is done to provide evidence about chemical weapons and support the United Nations and international law. – Angus Robertson M.P. Scottish Nationalist Party, Westminster.
One thing though is desperately important. The evidence must not have been tampered with. The elephant in the room, spoken or unspoken but uppermost in the thoughts of the JIC, the politicians and everyone else, is the dreadful example of last time round, in Iraq, when a document presented to Parliament supposedly based on intelligence reports said that that Saddam Hussein could deploy weapons of mass destruction at 45 minutes notice – and Parliament voted on the basis of that information. One honest and unfortunate man was quoted as saying that the document had been ‘sexed up.’ He killed himself when the full competing weight of media and government descended upon him. It turned out, after the war was over, that the document was indeed completely inaccurate. Saddam Hussein had no weapons of mass destruction at all, and had long since abandoned any serious attempt at developing them.
Honest data is primary. Without that, politicians cannot hope to make the right decisions, no matter how hard they try.
