Every team into Nome so far said this year’s race was unusually challenging on a number of fronts, injuries, snow-less trail conditions, and storms. The mushers are impressive, for sure, but you can’t say enough about their dogs.
Moving right along…
There was a report in the news recently around a paper published in Nature Medicine regarding a blood test that could predict whether or not an elderly person would develop Alzheimer’s related dementia (ALZ)*. Setting aside the obvious question of why anyone would want to know in advance if they were likely to get ALZ, the publication had other problems. Most notably, the authors claim that the blood test is 90% accurate. Meaning, given a blood draw from 10 elderly people (70 or older), all of whom let’s say would in fact go on to develop ALZ, the blood test would correctly identify 9 of them, and incorrectly conclude one of them would not go on to develop ALZ. In statistical lingo, the former are known as “true positives”, and the latter one a “false negative.” The test is based on the level in the blood of ten independent lipid molecules (fats) as measured by sensitive mass spectrometry, which, having considerable experience with it, appeared to me to be expertly done. In addition, how they arrived at those specific ten lipids was I think sound, but in any case unimportant to the following criticism.
Which was nicely and clearly explained here. If you follow it, you’ll see that even under the most generous estimate using the paper’s author’s own numbers, the best accuracy the test could hope to achieve is 62%, not 90%. Using more realistic calculations, the accuracy falls to only 32%.
An even bigger concern if you ask me is what the author at that link pointed out, that even assuming a 90% accuracy rate, if you performed the test on 1000 random patients only 50 of whom would actually go on to develop ALZ (ALZ is estimated to have a 5% prevalence in the population), then 0.9*50 = 45, and 0.1*950 = 95, meaning the test would correctly identify 45 of the 50 (true positives) people who would in fact develop ALZ, but incorrectly identify 95 people predicted to develop ALZ (false positives). Even being generous and using the more likely accuracy of 62%, true positives = 31 and false positives = 361. Which would mean 361 people left to believe they’re destined to get ALZ.
I am beginning to understand and appreciate the public’s skepticism around many of the du jour medical claims being made lately. How this study got published in Nature Medicine is truly puzzling to me.
* Hat tip Derek Lowe @ In the Pipeline