How many times have you heard a claim like: Doing X increases your risk of getting disease A by 75%? Or how about a claim like: Taking supplement Y cuts your risk of getting a disease B in half? Such statements of relative risk can be very misleading. Here’s why.

In 1995, Britain’s National Committee on Safety of Medicines issued the following warning to women taking contraceptives: Rigorous studies have found that women taking 3rd generation contraceptives (contraceptives made after 1990) experienced a twofold (100%) increase in blood clots compared to a similar cohort of women who took 2nd generation (pre 1990s) contraceptives. In other words, newer contraceptives are doubling a woman’s risk of blood clots compared to older contraceptives.

It sounds like young women should avoid 3rd generation contraceptives because of the increased risk of blood clots, right?
Wrong.

Here are the hidden, yet important details on prevalence:

In reality, the contraceptive studies found that one out of every 7,000 women who took the 2nd generation pills had blood clots. This number doubled with 3rd generation pills where two out of every 7,000 women who took 3rd generation pills had blood clots. Hmm? So is a 100% increase from 1 to 2 blood clots in a sample of 7,000 women something to get deeply concerned about? Not likely.

If the prevalence rate of a disease is low, then a 2, 3, or 4 fold increase may not be important, unless you are among the few who get the disease. 

This fallacy also works its deceptive magic the other way – in preventing disease.

What if I told you that vaccine A cuts your child’s risk of getting disease X in half? Hmm? Sounds good, eh? Wait a minute. What is disease X’s prevalence rate? The prevalence rate/risk of getting disease X is 2 in 10 million, or 0.00002%. Now if your child is vaccinated, the risk of getting the disease is cut in half to 1 in 10 million, or 0.00001%. Does this sound reasonable? Probably not. Think about it this way: we would need to vaccinate 10,000,000 children in order to prevent one child from getting disease X.

Should we spend millions to vaccinate children against a disease most will never get? To do so might be a waste of healthcare resources and dollars that could be utilized elsewhere. 

So the next time you read or hear a result like “It increases the risk of disease X by 200%” or "It cuts the likelihood of developing disease X in half," look for the prevalence rate. As the prevalence in the population goes up, so does the significance of the risk. For instance, if the prevalence rate is 10% (10% of people are at risk for developing disease X), then a vaccination that cuts that risk in half to 5% is something to crow about.

(Source: Gigerenzer et al. (2008). Helping doctors and patients make sense of health statistics. Psychological Science in the Public Interest, 8(2), 53-96.)
 


Comments

Joseph Smidt

Tue, 25 Aug 2009 2:10:27 pm

Really interesting analysis. It is amazing what people do with "true" statistics. That was a really interesting example.

Moreover, though they may have worked with 7000 real people, they are only a sample of the whole society. This means when applying these numbers to the whole world there is error.

In fact, the error on such an experiment probably means that both drugs are just as likely to cause blood clots since 1 in 7000 is only one sample. It is completely reasonable to believe if they tried it again on 7000 new women 2 would get the blood clots.

Thanks for pointing this out. Next time drug people make claims I am obviously going to have to dig deeper.

 

Tim

Tue, 25 Aug 2009 2:21:00 pm

I'm guessing that more than 7000 women were tested in the study.
I guess my questions would be--is there a real advantage to the third generation birth control? Are we just talking about clots, or are we talking about clots leading to more serious problems? Are the women being warned that the drugs they are taking are potentially more likely to harm them than alternative drugs? Are they aware of the risk?
As far as vaccines go--as long as everyone is getting vaccinated, the disease can't spread. I agree that some vaccines may not be necessary, but the low infection rate may not continue if people stop getting vaccinated for it. Some major diseases that we've been vaccinating against for decades have made minor comebacks among groups of people who refuse to be vaccinated. It's a "tragedy of the commons" problem--the cost of vaccination means the individual may be better off not vaccinating, but if everyone stopped vaccinating, there's a good chance the disease will make a major comeback and wreak some deadly havoc. It may be worse for the individual to vaccinate, but it's better for mankind if everyone vaccinates.

 

Dave C.

Tue, 25 Aug 2009 2:38:19 pm

Joseph,
Good point on the small numbers issue. Bias is always a potential factor when small numbers are involved. Given that the rates per 7000 were just 1 and 2, it is possible that the 3rd generation pills are safer than the 2nd generation pills. It possible that in the 3rd gen. pill studies, the researchers' samples happened to include 2 women with a biological disposition to clots. If they weren't in the study, the 3rd gen. pill cohort would have had zero clots. Conceivable, I think.

 

Dave C.

Tue, 25 Aug 2009 2:49:31 pm

Tim,
There were 4 studies on 3rd gen. pills, all reaching similar conclusions. The 3rd gen. pill is a slightly different formulation that is supposed to maximize benefits not related to birth control (e.g., reduce acne and prevent ovarian cancer) and minimize side effects.
You raise a good point on certain infectious diseases. The prevalence rates are very low, but only because we are vaccinating for those diseases. In such cases it may be better to talk about the potential for outbreaks and contagion rather than prevalence rates.
I know a chiropractor who is adamant about not vaccinating his kids. I am sure he has boasted on more than a few occasions that his kids have never gotten the diseases for which we vaccinate for, so therefore vaccinations are not necessary. Of course, his kids are benefiting from everyone else getting vaccinated.

 

Tim

Tue, 25 Aug 2009 3:27:09 pm

Dave,
Any stats on how well the third generation pill reduced acne, prevented ovarian cancer, or reduced side effects? As far as clots go (which, in my understanding, is the most serious side effect of some birth controls), the third generation pills failed (at least according to those 4 studies).
You are right to point out that "double the risk" doesn't necessarily mean the risk is substantial, especially if the benefits outweigh the added risk.

 

Stan

Tue, 25 Aug 2009 7:22:58 pm

In game theory those that don't vaccinate are called defectors. They don't take the risk (however low or imagined that may be) of getting vaccinated but they reap the benefit of the herd immunity. When the balance of defectors leans too far the other way there is a correction.

 

djinn

Wed, 26 Aug 2009 10:11:53 am

You say: "Should we spend millions to vaccinate children against a disease most will never get? To do so might be a waste of healthcare resources and dollars that could be utilized elsewhere."

The reason that no one is getting the disease (in the case of pretty much all vaccines) is because everyone is vaccinated. Once people stop being vaccinated, then the risk goes way up. Look at Great Britain, where measles is again becoming a problem, or the mini measles epidemic in San Diego a couple of years ago. As Stan mentioned the vaccine/disease ratio has two (2) TWO variables--your chance of getting the disease is dependent upon the number of people being vaccinated. Get the vaccinations, unless you want to risk your kids' (or other children's) health. A baby almost died in San Diego (prior to its vaccination date) because it was exposed to some kid whose selfish parents didn't get him vaccinated.

 

Dave C.

Wed, 26 Aug 2009 10:51:06 am

Djinn,

I adressed that concern in a previous comment. I agree with your argument when talking about things like Measles, Mumps, Pox, Rubella etc. However, the argument does not apply in all situations. For instance, there would be no need to vaccinate somone in the USA who does not travel abroad for a disease that is only common in third world countries. I am talking about diseases related to insufficient hygiene, etc.

 

Comments are closed.
Google Analytics