Calculated Risks

 

http://www.bklein.de/statistics/700Literature/gigerenzer.pngThis article is reprinted with permission from the August 13, 2002 edition of the New York Law Journal. © 2002 NLP IP Company. All rights reserved. Further duplication without permission is prohibited.  

Calculated Risks: How to Know When Numbers Deceive You

By Gerd Gigerenzer

Simon & Schuster, New York, N.Y., 320 pages, $25

Reviewed By Phil Schatz
New York Law Journal

Only 1 of approximately 175 accredited law schools requires a course in basic statistics or research methods. As Gerd Gigerenzer, director of the Max Planck Institute for Human Development in Berlin , points out in his penetrating new book Calculated Risks: How to Know When Numbers Deceive You, the statistical innumeracy of lawyers and lay people makes us gullible and prone to uninformed decisions.

 Are you statistically challenged? This reviewer is, although less so after having read Dr. Gigerenzer’s book. Here is a quick quiz to find out:  

  1. What percentage of drivers are better than average?

 Around 63%, when “average” is determined by number of accidents. This is so because the distribution of accidents is asymmetrical; bad drivers account for more accidents than good ones, so most drivers have fewer than the average number of accidents.

  1. If men with high cholesterol have a 50% higher risk of heart attack than men with normal cholesterol, should you panic if your cholesterol level is high?

Probably not. Although 50% sounds frightening, it is only because it is given in relative terms: 6 out of 100 men with high cholesterol will have a heart attack in 10 years, versus 4 out of 100 for men with normal levels. In absolute terms, the increased risk is only 2 out of 100 – or 2%. Look at it this way: Even in the high cholesterol category, 94% of the men won’t have heart attacks.  

  1. HIV tests are 99.9 percent accurate. You test positive for HIV, although you have no known risk factors. What is the likelihood that you have AIDS, if 0.01 percent of men with no known risk behavior are infected?

Fifty-fifty. Most people assume the possibility is much higher, an illustration of the “illusion of certainty.” The correct answer is clear if the problem is framed in frequencies: Take 10,000 men with no known risk factors. 1 of these men has AIDS; he will almost certainly test positive. Of the remaining 9,999 men, 1 will also test positive. Thus, the likelihood that you have AIDS given a positive test is 1 out of 2. A positive AIDS test, although cause for concern, is far from a death sentence.  

  1. The blood found under the fingernails of a murdered woman matches the defendant’s blood type, which only 17.3 percent of the population shares. The blood on defendant’s shoes matches the victim’s type, which only 15.7 of the population shares. An expert witness at trial testified that multiplying these two probabilities together gives a joint probability of 2.7 percent that these two matches would occur by chance – and that there was, therefore, a 97.3 percent chance that defendant is the murderer. What is the flaw in the expert’s reasoning? 

This is an example of the “prosecutor’s fallacy” – namely, the erroneous assumption that the random match probability equals probability of guilt. The actual possibility that the defendant is the murderer based solely on these two matches is very small. Frequency analysis again shows why: Assume that any of the 100,000 men in the city where the murder took place could be the murderer. One of them, the murderer, will show both matches with practical certainty. Of the remaining 99,999 other residents, we can expect that 2,700 (2.7%) show the same matches. Thus, the probability that a man with both matches is the murderer is 1 in 2,700 - less than one-tenth of 1 percent..  

  1. In his argument to the court to exclude evidence that O.J. Simpson had battered his wife, Alan Dershowitz successfully argued that the evidence was irrelevant because, although there were 2.5-4 million incidents of abuse of domestic partners, there were only 1,432 homicides. Thus, he argued, “an infinitesimal percentage – certainly fewer than 1 of 2,500 – of men who slap or beat their domestic partners go on to murder them.” Dershowitz’s argument is outrageously incorrect: the actual likelihood that a batterer actually murdered his partner is 8 out of 9, or around 90%. What is missing from Dershowitz’s analysis?  

 Either Dershowitz was confused, or he purposely hoodwinked the court, in much the same way that the tobacco industry seeks to obscure the risks of smoking. His analysis omits a key element: what number of battered women are killed each year by someone other than their partners? The answer is around 0.05%. Now, think of 100,000 battered women. 40 will be murdered this year by their partners. 5 will be murdered by someone else. Thus, 40/45 murdered and battered women will be killed by their batterers -- in only 1/9 cases is the murderer someone other than the batterer. 

 -----

Even though most non-statisticians find these sort of questions difficult, they have no trouble understanding the answers. Statistical thinking is learnable. A central proposition of Gigerenzer’s book is that presenting statistical information in natural frequencies (e.g., 12 out of 100), as described in practice under questions 4 and 5, would greatly increase the average person’s understanding of and ability to make informed decisions about risks. (A British Court of Appeal recently recommended that DNA evidence be presented in a frequency format). The book is punctuated throughout by compelling examples that demonstrate the abuse of statistics in the medical and legal worlds – often purposefully to exploit statistical innumeracy so as to get funding for dubious research tasks, to sell a particular mode of treatment by raising anxiety, or to manipulate profit and loss statements. Some of the examples are humorous and obvious -- a municipality repainted a four lane highway to add two lanes, erased the extra two lanes when accidents dramatically increased, and then argued that it had increased road capacity by 17 percent (from four to six lanes was a 50% increase; from six to four was a 33% decrease)!  Some are deadly serious and likely to raise serious outcry from vested interests, such as Dr. Gigerenzer’s analysis of the cost-benefits of prostate and breast cancer screenings, and the reliability of DNA and fingerprint tests.

 This is a serious and important book. Although it covers some of the same ground as Paulos’s Innumeracy and Huff’s How to Lie With Statistics, it is more topical and pointed, and truly is must-reading for anyone who is not completely at home with statistics.

Phil Schatz is a member of Wrobel & Schatz.