Death By Bias

I've gradually become addicted to Overcoming Bias, and specifically Eliezer Yudkowsky's contributions to it. And it struck me, reading the reports on the de Menezes trial, that a good dose of this blog could have done the Metropolitan Police a power of good.

Specifically, members of the police command staff recalled hearing a radio message first that de Menezes was definitely not the suspect, and then that he definitely was the suspect. Now, it's very unlikely indeed that someone who was behaving rationally would go from certainty of X to certainty of Y without passing through stages of progressively greater doubt about X. It's possible that you might encounter a situation when you had strong enough evidence to do the whole leap; it's just very, very unlikely, and therefore you should be suspicious of any such suggestion. There's a good reason for the cultural norm that you should be suspicious of sudden converts' motives.

Similarly, it's very hard to imagine a scenario where the police could have gone from being certain that he definitely wasn't a suicide bomber - a prior of zero - to certain that he definitely was. Either they weren't certain to begin with, in which case the officer in question shouldn't have said so, or they weren't certain when they changed their mind, in which case they doubly shouldn't have said so.

In fact, the account of the police command-and-control of the operation that emerges is an appalling hellbroth of cognitive bias. The specialist firearms team was briefed in terms described as "inflammatory" about "firing a bullet into the brain of a suicide bomber", immediately after having been issued with special 124 grain ammunition - had it been calculated to embed a perceptual fix that whoever they ended up chasing was indeed a suicide bomber, it couldn't have done so better. Under stress, people tend to exhibit perceptual rigidity, blocking out information, and target fixation.

Further, for reasons that still haven't been made clear, this outfit didn't reach the scence until 5 hours after the original call for them; so there was no time for them to be cross-briefed by the surveillance team. So lacking in orientation, and cranked up with aggression and tension, were they that one of them chased the train driver into a tunnel waving a gun under the impression he was another member of "the cell". But the surveillance squad had followed only one man into the station. Where did this cell come from, other than crisis fever and ignorance?

This general farrago of stupidity was matched at headquarters, where the command centre was besieged by every other staff officer who could squeeze in to watch the fun; the hubbub was such it was difficult to hear the radio traffic, and tempers can only have been wearing out - another risk factor, as was the fact everyone had been up all night. Worse, it seems there may have been two commanders - everyone remembers Cressida Dick, who was given out until recently as having been the Gold Commander, but it appears that this title was also held for much of the operation by John McDowell. It is not clear whether they were co-equal (pretty bad) or whether there was a change of command in the middle of the crisis (even worse).

The overwhelming impression is that the Met is not serious about the super-duper war-on-terrorism role various chiefs, especially Sir Ian Blair, have been so keen to take on and expand. Its command arrangements here were desperately bad, to say nothing of Blair's statements in the aftermath or the leak campaign against the victim. Then, there is the Forest Gate incident when a man was shot "by accident"; the military call this a "negligent discharge" and treat it very seriously indeed even if no-one is hurt or anything damaged, but the Met essentially shrugged it off and contented itself by leaking to the NOTW that the victim was a paedophile.

It is not enough to blame the pilot; it is not enough to say it was an accident. The only conceivable answer to this would be along the lines of "If that's your best...I don't want to see your worst." Accidents happen for reasons; reasons that are found in institutions. This is a sick institution.