On the culture of getting paid for work

My latest blog post for the ACS Chemical Biology Community in the ACS Network is up, on the culture of getting paid for work.

How do you decide what work you should be paid for (and pay for), and what work should be an unpaid extracurricular, hobby, or favor? What kind of compensation besides money would you accept or have you accepted for work?  Are unpaid internships “engines of inequity” that exploit young people, or they a good way to help young people get real work experience before they know enough to actually contribute to a company’s success?

The Graduation Ceremony Paradox

My graduation ceremony was on Thursday, and I blogged about it for the ACS Chemical Biology Community on the ACS Network.

VAJC, week 2: aluminum adjuvants, ASD, and Hill's criteria (Tomljenovic and Shaw)

-->

Welcome to the second installment of the Vaccine Autism Journal Club project!

Today we’ll be discussing “Do aluminum adjuvants contribute to the rising prevalence of autism?”, by Lucija Tomljenovic and Christopher A. Shaw, Journal of Inorganic Biochemistry 105:1489-1499 (November 2011).

Disclaimer:  I am not an epidemiologist or statistician.  If you have expertise in these fields, and I get the analysis completely wrong, please let me know in the comments!

Question this paper is trying to answer:  Does the increasing use of aluminum adjuvants in vaccines meet Hill’s criteria for causation of autism?

 

 

Background:  An adjuvant is a substance added to a vaccine to increase the body’s immune response to the vaccine.  In the United States, adjuvants contain aluminum.  You can read the CDC’s frequently asked questions about adjuvants here.

 

In 1965, Sir Austin Bradford Hill proposed a set of criteria to consider when trying to decide whether the most likely interpretation of an association is causation, especially in the context of environmental factors and disease.

 

 

Methods:  The authors gathered data for autism spectrum disorder (ASD) prevalence from several countries.  They also gathered data on the recommended vaccine schedule for each of those countries, and the amount of aluminum that a child would be exposed to if they followed the schedule.  Different brands of vaccines contain different amounts of aluminum, so they calculated the minimum exposure (if you followed the schedule using the brands with the least aluminum), the maximum exposure (if you followed the schedule using the brands with the most aluminum), and the average exposure.  The average is quite literally an average: if there are three brands of DTaP vaccine, they add up the aluminum content and divide by three.  This method of averaging seems silly to me: just because there are three brands doesn’t mean that each brand holds exactly one third of the market.  But as we will see, this is hardly the greatest weakness of the paper, so I’m not going to dwell on it.

 

The authors also assume that aluminum exposure accumulates: i.e., that the human body eliminates none of the aluminum, even over the course of months and years.  Their main justification for assuming an elimination of zero is that “Al [aluminum] has been shown to persist at the site of injection from several months up to 8-10 years following vaccination in patients suffering from macrophagic myofasciitis, an autoimmune disease linked to Al vaccine adjuvants”.  MMF is an extremely rare disease characterized precisely by an abnormal response to aluminum adjuvants.  The idea that you would use aluminum metabolism in these patients as a way to estimate aluminum metabolism in people without MMF is crazy.

 

The authors then look at how aluminum exposure over time correlates with ASD prevalence over time.

 

 

Results:  The authors find an excellent correlation between aluminum exposure (which has been increasing in recent years) and ASD prevalence (which has also increased in recent years), in the population as a whole.

 

The authors go on to claim that the strength and consistency of the correlation, plus the appropriate temporal relationship between the proposed cause and outcome (children are vaccinated as infants; ASD symptoms appear later), plus what they call “a biologically plausible mechanism by which it can be explained”, all meet Hill’s criteria for causation, causing them to conclude that aluminum adjuvants should be seriously considered as a cause of ASD.

 

 

Discussion:  Correlation does not imply causation.  Correlation doesn’t even imply a relationship.  The famous example the great evolutionary biologist and popular science writer Stephen Jay Gould gives in his book The Mismeasure of Man is that he can find an excellent correlation between his age, the population of Mexico, the price of Swiss cheese, his pet turtle’s weight, and the average distance between galaxies.  The true relationship between all of these variables, of course, is that they’re all increasing with time.  To conclude that the reason the average distance between galaxies is increasing because Stephen Jay Gould’s pet turtle is gaining weight, and to propose that putting the turtle on a diet will cause the galaxies to move closer together, is insane.

 

Similarly: the prevalence of ASD has increased in recent decades.  You will find an excellent correlation, therefore, between ASD prevalence and anything else that has also increased in recent decades: not just aluminum adjuvants, but also organic food sales, car seat use, and, I dare say, my age, the population of Mexico, the average distance between galaxies, and the weight of a living pet turtle.  All that should be concluded from these observations is that all of these depend on time, not that there’s any causal relationship between any of them.

 

 

The authors claim that the correlation between aluminum adjuvant exposure and ASD prevalence meets several of Hill’s criteria, the first being the strength of the correlation.  It is true that the authors’ confidence in the correlation is quite high (p < 0.0001): but that is not what Hill meant by the strength of the correlation.  The example Hill gave in his remarks to the Royal Society of Medicine is as follows:

 

prospective inquiries into smoking have shown that the death rate from cancer of the lung in cigarette smokers is nine to ten times the rate in non-smokers and the rate in heavy cigarette smokers is twenty to thirty times as great. On the other hand the death rate from coronary thrombosis in smokers is no more than twice, possibly less, the death rate in non-smokers. Though there is good evidence to support causation it is surely much easier in this case to think of some feature of life that may go hand-in-hand with smoking – features that might conceivably be the real underlying cause or, at the least, an important contributor, whether it be lack of exercise, nature of diet or other factors. But to explain the pronounced excess of cancer of the lung in any other environmental terms requires some feature of life so intimately linked with cigarette smoking and with the amount of smoking that such a feature should be easily detectable.

 

When Hill talks about the strength of the association, he doesn’t mean the confidence: he means the magnitude in the increase in risk.  In order to say that the correlation between aluminum adjuvants and ASD prevalence meets Hill’s criterion of strength, then the authors would have to show that an x increase in aluminum adjuvant exposure leads to a y% increase in ASD risk, where y is some large number.  The authors, of course, cannot do that with the data they’ve collected, because they’re not comparing children with different amounts of aluminum exposure: they’re just looking at entire populations over time.

 

A much better analysis would have been to compare children who had gotten the minimum dose of aluminum adjuvants (based on the brands of vaccine they happened to be vaccinated with) to children who had gotten the maximum dose of aluminum adjuvants, and to see whether the risk in ASD differed between these two groups of children.  The authors, of course, did not do that analysis.  In fact, this paper has no controls at all.

 

 

The second of Hill’s criteria the authors claim that the association between aluminum adjuvants and ASD prevalence meets is consistency.  I’m not entirely sure what they mean: the best I can come up with is that they were able to find a correlation between aluminum adjuvant exposure and autism not just in the US, but also in the UK, Canada, Australia, Sweden, Ireland, and Finland.  But once again, repeating the same method in multiple data sets and getting the same result is not what Hill meant by consistency.  According to Hill,

 

the same results from precisely the same form of inquiry will not invariably greatly strengthen the original evidence. I would myself put a good deal of weight upon similar results reached in quite different ways, e.g. prospectively and retrospectively.

 

This paper does not use “quite different ways” to try to find a correlation.  Indeed, even a single paper cannot provide Hill’s criterion of consistency, which would be an affirmative answer to “Has it been repeatedly observed by different persons, in different places, circumstances and times?”, because a single paper cannot be observed by “different persons”.  Only if the result is found by multiple groups of researchers can it really be said to be consistent.

 

 

I could go on about the other criteria they claim, but I think you get the point.

 

 

Conclusion:  This paper is completely bogus.

 

 

Please leave your comments and questions below.

Our next paper will be “A positive association found between autism prevalence and childhood vaccination uptake across the U.S. population”, Delong, Journal of Toxicology and Environmental Health, Part A 74:903-916 (May 2011).

 

I will be away from my computer during the next couple of Fridays, so I will tentatively schedule the next installment of the Vaccine Autism Journal Club project to appear on or about May 23.  See you then.

The importance of industrial experience

My latest post for the ACS Chemical Biology Community on the ACS Network is up, on commonly cited reasons why industrial experience is key, and why I think my academic research experience addresses those points.