Lies, damn lies, breast cancer, and statistics

One of the most infuriating things about statistics is that one can generally find a statistic to support whatever claim one is trying to make. In many cases, this isn't necessarily a matter of dishonesty, but merely a choice one makes about which statistics to include, or which method of calculation one uses. This point was really brought home to me while reading a post from Orac on the benefits or lack thereof of mammography.

The issue at question is whether or not regular mammography screenings for women in their 40's with no risk factors for breast cancer are beneficial in terms of detecting cancers early and thus allowing women with breast cancer to live longer. Orac's post has an enormous amount of detail on this debate (in his real life, he is a clinical oncologist specializing in breast cancer), but the issue I want to focus on is a passage in which he discusses the impact regular mammography screenings have on reducing the risk of death from breast cancer. He writes,

Woloshin and Schwartz also point out that for women between the ages of 40 and 49, mammographic screening is associated with a reduction in risk of dying of breast cancer over 10 years from 0.35% to 0.30% (a relative risk reduction of 14%); between the ages of 50 and 59, from 0.53% to 0.46% (a relative risk reduction of 13%); and between the ages of 60 and 69, from 0.83% to 0.56% (a relative risk reduction of 33%). When examined on a relative basis, a risk reduction of 13 to 33% looks impressive. However, when examined on an absolute basis, these risk reductions sound a lot less impressive. It is generally a truism in medicine that, if you want to make the apparent benefit sound as good as possible, you use relative risk reductions but that if you want to make relative risk reductions sound as unimpressive as possible you use absolute risk reductions. That’s because absolute risk reduction incorporates the risk a person has of developing the condition being intervened against. This is the same issue that comes up when discussing improvements in five year survival brought about by chemotherapy in cancer. For instance, I can tell you that, if you have a stage I cancer, chemotherapy will improve your five year survival by 30%. That’s a relative number. However, if I cite it in terms of absolute risk reduction (rounding to make the numbers easy), it is a 3% absolute improvement in survival (from 90% to 93%). Personally, I believe in quoting both figures.

The key take away from this is that one can take the same piece of data, the reduction in risk of dying from breast cancer with regular mammography, and present that information in two different ways that could support two diametrically opposed conclusions. If one wants to argue that mammography is not very useful, then one can use the reduction in absolute risk, say from 0.35% to 0.30%, a reduction that appears insignificant. By contrast, if one wants to argue that mammography is beneficial, one can focus on relative risk and say that mammography reduces the risk of death from breast cancer by 14%. In either case, one is accurately reporting the same information, but one is looking at it in different ways, the difference between absolute and relative risk reduction, and potentially reaching different conclusions.

So, statistics can be illusive, but they are also extremely valuable tools for taking a great deal of data (the death rates from cancer in the US) and making it easily accessible and comprehensible. As critical thinkers, however, we must exercise caution when using and discussing statistics to make sure that we are being honest and that we are not being taken in by dishonest manipulations of statistics. We must make sure we know exactly what the statistics we are confronted with actually mean as well as how they were calculated, and we most always look and see if there are other ways to accurately present the information in question. I find Orac's strategy of presenting both figures to be the right way to go.