99.9% Certainty It Was OBL – Really?

Undoubtedly you’ve read by now one or more reports that the man killed in Pakistan was, with 99.9% certainty, Osama bin Laden (OBL). You do not have to be a conspiracy nut or deeply skeptical of any government claim to be critical of this particular claim. I happen to believe the man killed was very likely OBL, a claim importantly different than the one stated – 99.9% certain. Reading people’s comments to the various news reports, many appear confused or incredulous over a couple things:

1. If the government did not already have a DNA sample from OBL, what did they compare the sample from the dead man to?

Most likely his (putative) sister [1,2]. The most common DNA test to perform in this situation is one which compares specific portions of the nuclear DNA extracted from the samples, portions called “short tandem repeats” (STR). Without going into detail, the more related two people are the more similar their STRs will be. Comparing the STRs from the dead man’s sample to the STRs of the (putative) sister would allow one to make a claim of the likelihood of kinship of the two people, but not a claim of x% certainty of identity of the dead man. That’s different:

The key distinction, Thompson says, is that the results speak to the probability of a relationship between the two people, not the actual identity of the dead man. “If the DNA test compared the profile of the man shot in Pakistan with bin Laden’s family members,” Thompson says, “the results could properly be presented only as a likelihood ratio, stating the relative likelihood of observing the particular markers found in the dead man if he were, say, the father or brother of a known bin Laden family member, than if he were a randomly chosen individual.”

2. A DNA test could not have been performed this rapidly – in less than 24 hours of the killing the government had made its claim of 99.9% certainty of identification.

I don’t doubt the test could have been performed this quickly. Preparing a sample for that type of test and performing it (see PCR) could be accomplished in under 24 hours in a competent lab equipped with a modern sequence analyzer. It seems to me reasonable that the lab would have needed to be told in advance to get ready in order to complete it this quickly, but that is by no means impossible or even unlikely. After all, this operation had been planned for some time.

Expanding a bit on point #1: One can convert transform a likelihood ratio into a probability. Statisticians especially regard the two measures as technically different. In order to do this you need to quantify other prior evidence of identity and/or relationship. For example, comparing a photograph of the dead man to a previously obtained photograph of OBL could be one basis for such a prior probability, something clearly the Seals who did the killing could have done. Another possibility is that the lab also compared mitochondrial DNA (mtDNA, different from nuclear DNA) from the dead man and the putative sister. Since 100% of everyone’s mtDNA is inherited solely from their mother, then if the mtDNA from sister and dead man matched that too could be used as prior evidence. As could a Y-chromosome comparison between the dead man and a (putative) son; evidently OBL had many. The reports I read did not provide the details of the actual tests performed.

Performing all this analysis in such a short period of time would be very challenging, and necessary to achieve a statistical estimate of %certainty of identity, which is why I am dubious of this specific claim. But given the tests which were most likely performed in this short period of time, I can easily believe the man killed was very likely OBL.

Update: Inter-sample contamination is also a concern when performing DNA tests, but I am not so cynical as to think that this was a problem in this case.