Things that frost my shorts – more bad science edition (plus short stats lecture)

This blog post  came out about drinking coffee, an activity whose importance to me cannot be underestimated. I found it at the Smithsonian which presented the results in a headline smashing way. 

First, the Smithsonian version. The get the lede (mostly) right:

When it comes to coffee, the main piece of biology to consider is your body’s level of cortisol—a hormone related to stress and alertness.

It may not be the main, but one of the main,  considerations. But they screw it up with the “perscriptive phase” of the article, which is a direct quote from the blog piece:

Although your cortisol levels peak between 8 and 9 AM, there are a few other times where–on average–blood levels peak again and are between noon to 1 PM, and between 5:30 to 6:30 PM. In the morning then, your coffee will probably be the most effective if you enjoy it between 9:30 AM and 11:30 AM, when your cortisol levels are dropping before the next spike.

Let’s go to the blog piece. Its the home of one NeuroscienceDC. His article does cite the primary sources. It still contains the same “overall” conclusions:
This is because cortisol production is strongly related to your level of alertness and it just so happens that cortisol peaks for your 24 hour rhythm between 8 and 9 AM on average (Debono et al., 2009).

Here are the two cites (with extra points for having links):

Debono M, Ghobadi C, Rostami-Hodjegan A, Huatan H, Campbell MJ, Newell-Price J, Darzy K, Merke DP, Arlt W, & Ross RJ (2009). Modified-release hydrocortisone to provide circadian cortisol profiles. The Journal of clinical endocrinology and metabolism, 94 (5), 1548-54 PMID: 19223520

Inouye, S.T., and Kawamura, H. (1979). Persistence of circadian rhythmicity in a mammalian hypothalamic “island” containing the suprachiasmatic nucleus Proceedings of the National Academy of Sciences of the United States of America DOI: 10.1073/pnas.76.11.5962

But the blog post is still full of “your levels spike at…” kind of general advice. For me this is a consistent problem with lots of neuroscience: we don’t need no stinkin’ statistics, data analysis, or conception of what inter-individual variation is.

Next stop: original article. From the original abstract (emphasis mine):

Using data from a cross-sectional study in healthy reference subjects (n = 33), we defined parameters for the cortisol rhythm. We then tested MR-HC against immediate-release hydrocortisone in healthy volunteers (n = 28) in an open-label, randomized, single-dose, cross-over study. We compared profiles with physiological cortisol levels, and modeled an optimal treatment regimen.

Now, for a human study, 33 is probably a nice healthy sample size, if the data do not need to be stratified, or there is no obvious co-variate. What does stratification/co-variate mean? It means that there isn’t some other hidden variable screwing up your results. For example, that there are not subgroups. If your sample had males and females, 3rd shift workers, 2nd shift workers, and moms with newborns, you might have a few groups that were not given to “normal sleep”. You would have to include a variable that identified this group in your analysis. A co-variate is a continuous variable that might effect your answer.  If you were measuring weight loss (absolute number of pounds) before and after a drug, you would want to include starting weight as a covariate, on the basis that a 250lb person losing 15 pounds is a lot different from a 100lb person losing 15 pounds.

This original study seemed to have several sub=studies. One used a paired/repeated measures study where they had detailed individual data, and compared each individual to his/her own baseline. Others did not. Nowhere is mentioned any correction or stratification for chronotype. Maybe they didn’t believe that such exist. But this article wasn’t about coffee. It was about rhythm of cortisol.

The addition of coffee seems to be the new contribution of our intrepid correspondent. It is an interesting article trying to bring science to the coffee-drinking masses (always a good thing). But I think the specific recommendations about when to consume your caffeine are not going to be valid unless your body rhythms follow normal. I think there could have been more qualification of the results, which of course makes it a bit less sensational. But, probably more correct. I am sure my cortisol levels are not peaking at that time. I’m sure mothers with 6-week old infants have massively variable cortisol levels.

Bottom line: YMMV. A whole fucking lot.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s