Imagine if you will that you make a living by selling an idea. You are a merchant in the marketplace of ideas, and you sell an idea that we will call Idea X. You sell books that promote Idea X, you give lectures across the country extolling the virtues and benefits of Idea X, etc. Turns out that Idea X is somewhat controversial but quite lucrative. You tried selling other stories and ideas in the past, but none have been nearly as profitable to you as Idea X. Espousing this idea has afforded you the ability to provide your spouse and children a comfortable living in these tough economic times. Imagine also that a scientific study was recently conducted by prominent researchers and the results are published in a prestigious scientific journal. As it happens the results of the study are pretty devastating to Idea X and contradict some of its main points. In the following days the study results are much-discussed in various media outlets, and some people begin to doubt the idea that you’re selling. The study results have left the more loyal adopters of your idea confused as to what to believe, and they beg you to respond to the study.
What do you do?
There are probably a few different choices you could make in this case.
- You could ignore the study and hope this bad news blows over soon. Then after the media gets bored of discussing the study results and moves on, you can try to resume preaching your idea to the people still adhering to your idea. As time goes on you can try to re-convert those who left Idea X.
- You could follow where the evidence leads and come out and say “I don’t want to mislead anybody. It looks like Idea X is not supported by the evidence. I was wrong, and I’m sorry.” Then you face the very tough challenge of trying to find another idea that is both as profitable as your last idea with the caveat that it is also evidence-based and grounded in reality. If you don’t find another profitable idea then you’ll have to move your family out of your nice neighborhood and into a lousier one while you work unappealing jobs to put food on the table.
- You could denounce the study and the scientists who conducted it. You could try and poke holes in the evidence and muster your greatest argument for why no one should believe the results. It may not be intellectually honest, but the truth must be an unfortunate casualty when you and your family’s quality of life is at stake.
Gary Taubes has chosen #3, except he takes it even further and denounces not only the specific study and the researchers that conducted it, but the field of epidemiology as a whole. This is actually pretty shrewd since most epidemiological evidence is against him anyway. Taubes’s Idea X in this analogy is the assertion that, contrary to popular belief, foodstuffs like red meat, saturated fat, and cholesterol are actually healthy and pose no threat of disease while just about any and all carbohydrates are unhealthy.
Mr. Taubes recently wrote a blog post responding to a recent study titled Red Meat Consumption and Mortality published in Archives of Internal Medicine. The study concludes “red meat consumption is associated with an increased risk of total, CVD, and cancer mortality.”1 If I had to summarize Mr. Taubes entire blog post it would be something like: Epidemiology is a pseudoscience. Well, not all epidemiological research, just any research suggesting that eating red meat for every meal might not be healthy. Any scientist or nutrition researcher claiming that is practicing a junk science. Those that come to other conclusions are the real scientists, the good scientists. Also, I will never accept any scientific claim unless there are randomized, placebo-controlled clinical trials to back it up. Except for the claim that smoking causes lung cancer. I will accept that one without RCTs.
Let’s Break it Down
To start things off Taubes does not refute any specific thing in the actual paper. He leaves that job to someone else.
Zoe Harcombe has done a wonderful job dissecting the paper at her site.
Taubes prefers instead to respond to the paper’s general tone and the tone of some of the researchers involved with the paper, notably Dr. Walter Willett and Dr. Meir Stampfer.
Who is this Zoe Harcombe and why should I trust her judgment? The biography on her webpage does not tell me much about her credentials other than she is a
[Q]ualified nutritionist with a Diploma in Diet & Nutrition and a Diploma in Clinical Weight Management, but she is first and foremost an obesity researcher.
A “qualified nutritionist”? What does that even mean? Who “qualified” her and gave her “Diplomas”? Were these actual universities or did she just pay 30 bucks (or pounds perhaps, since she’s from the UK) for some shady online certification that means nothing? We are not told, and I suspect for good reason.
She is also an obesity researcher, huh? A quick Google Scholar search of her name came up with no publications. Is she actually a researcher or does she just call herself one? I suspect the latter. I love how Taubes apparently thinks she is some sort of authority on the matter but the Harvard School of Public Health is full of incompetent boobs.
Here is one of her “key problems with this study”
1) This study can at best suggest an observed relationship, or association. To make allegations about causation and risk is ignorant and erroneous.
Nowhere in the study did the authors claim to have a causal relationship. It is always stated as an association. Harcombe misrepresents the results.
Eventually her “key problems” devolve into irrelevant and unverifiable ad hominem
[O]ne of the authors (if not more) is known to be vegetarian and speaks at vegetarian conferences
It is like the author is a communist or something! As if being a vegetarian would have any bearing on the results of the study. Are vegetarians not supposed to conduct nutrition research now, only omnivores? What about Jews or Muslims? If they do research involving meat should their results not be published? What’s more is this claim isn’t even verified. She has a citation but it takes me to this page that tells me nothing. I also went to the page with the 2013 speakers but none of them authored the study in question.
Okay, back to Taubes now.
The problem with observational studies like those run by Willett and his colleagues is that they do none of this [testing hypotheses]. That’s why it’s so frustrating. The hard part of science is left out and they skip straight to the endpoint, insisting that their interpretation of the association is the correct one and we should all change our diets accordingly.
You mean like how you do, Taubes, with your books and lectures? Except that you are not a scientist and have no medical training or research experience like Willett et al. I don’t want to engage too much in tu quoque, but pointing out hypocrisy is so much fun. Let me get in one more jab, and then I’ll move on.
I’m no expert in Taubesian hermeneutics, but I think he meant to say The problem with observational studies like those run by Willett and his colleagues is that they are so devastating to my position that everyone should eat truckloads of meat. Okay now I’ll get to the substance of his point.
Mr. Taubes seems to be accusing Willett et al. of laziness despite the fact that they churn out hundreds of studies that take decades to complete and are peer-reviewed and published in top-tier journals.2 The reason for this is because, according to Mr. Taubes, epidemiology is not a true science, and if Willett and his colleagues really wanted to study certain dietary aspects and their link to colon cancer mortality or cardiovascular disease mortality they have to conduct randomized, double-blind, placebo-controlled human clinical trials (I will abbreviate as RCTs). Anything less is not “good science.”
I’m not about to give you a lecture on what epidemiology is and why it is important. It may be a good subject for a future post, but for now I will say that it is a useful field when studying large populations, incidence and prevalence of disease, or when RCTs would be unethical or impossible.
Here is a visual reference to help understand the hierarchy of evidence a little better. At the top of the triangle you have the most rock-solid evidence that we have on a given subject and the bottom is the least valuable evidence.
As you can see, the cohort studies in question are not quite as good as RCTs, but they are as close as you can get. They are not meant to replace RCTs in evidence-based medicine, but they can be a good proxy when conducting a RCT would be impossible. Here’s another visual that explains each method.
Mr. Taubes asserts that the experts are wrong and red meat does not lead to any kind of mortality. He asserts this with barely any evidence and zero RCTs to back up his assertion, yet if anyone would disagree he demands they provide RCTs because Taubes will not accept cohort studies. Conveniently for him the RCTs he demands will almost certainly never take place. Let me explain.
Mr. Taubes seems to think that RCTs examining specific dietary constituents and their roles in disease mortality are low-hanging fruit. I’d like to see Taubes design one for, say, red meat and colon cancer. It would be worth a read just to see how he deals with the problem of blinding alone. Furthermore, what would be a good placebo? I get that one group would be assigned red meat every meal, but how do you get the control group to eat placebo meat without the subjects really knowing if they are in the meat-eating group? Surely people can tell the difference between a tofu steak and a genuine ribeye. Plus cancer is not something that you get only a few weeks into a feeding study. It takes decades to arise, so it would mean that a RCT would also have to take as much time. There are many more difficult study design issues Mr. Taubes would need to tackle before he could begin.
This is all assuming of course that the study would be funded (it almost certainly wouldn’t because it would be the single most costly RCT in history) and that it would pass IRB approval (it wouldn’t because the study itself would be unethical considering there is a fair amount of evidence that red meat does in fact lead to colon cancer3, Red Meat Consumption and Mortality notwithstanding).
Throw it Against the Wall and See if it Sticks
If you don’t buy into his Epidemiology Sucks theory Taubes also hurls some other arguments against the study hoping at least one of them will stick. One is that the increase in mortality from red meat eaters is only an increase of 0.2 so it’s basically nothing and you should just forget about it. Of course another way to present that increase is 20%. An increase of only 0.2 seems like such a small number compared to 20%, doesn’t it? What if I told you that drinking Generic Beverage That You Sort Of Enjoy everyday increases your risk of developing esophageal cancer by twenty percent? My guess is you would cut down on Generic Beverage That You Sort Of Enjoy, even if you enjoyed it.
Another argument Taubes uses is admitting the following:
[T]he people who avoided red meat and processed meats were the ones who fundamentally cared about their health
If I may paraphrase: Okay, sure, vegetarians are healthier than meat-eaters. But is it because of the meat? I say it’s because vegetarians are just generally healthier than meat-eaters. They are more health-conscious and they smoke less. It has nothing to do with meat! It’s simply the “healthy cohort effect”!
Are you sitting down? I hope so because I’m about to rock your world: I agree with Taubes here. This is a legitimate concern with epidemiological studies. If you are not careful with your study design and statistical analysis these kinds of things can bias your results. However, there are many ways you can adjust for things like this, and I think it goes without saying that if you have a poor study design or no adjustments then you don’t get published in the top tier journals in which Willett and Stampfer get published. Furthermore, sampling biases like the healthy cohort effect are stronger in smaller cohorts of 10 or 20, but when you increase sample size to 100 or 1,000 the effect becomes negligible. In the case of Red Meat Consumption and Mortality the authors use cohorts of 37,698 men and 83,644 women, so if they did their due diligence with the statistical analysis (and I’m not about to check their math, that’s what the peer-review process is for) if the effect is even present it should be so weak as to be nonexistent.
As it stands the authors of the paper in question did in fact adjust for smoking status as well as a host of other lifestyle factors.4
Watch now as Taubes removes any possible hope you might have about his scientific literacy.
So do an experiment to see which is right. How do we do it? Well you can do it with an N of 1. Switch your diet, see what happens.
The ol’ do-it-yourself randomized controlled trial. The very pinnacle of oncological research. Have you switched to a high-beef, low-carb diet? Check. Have you died from colon cancer and/or cardiovascular disease? No? Well then the experts were wrong! Help yourself to some more mutton. Take note future lawyers: this is special pleading at its finest.
Credit Where Credit is Due
Although completely irrelevant to a discussion of Red Meat Consumption and Mortality Taubes does cite one randomized trial (not placebo-controlled) that kind of shows some benefits in a low-carb Atkins diet over other somewhat popular diets such as Zone or Ornish. It is totally a red herring, but let’s touch on it anyway. The study takes premenopausal women that are either overweight or obese and assigns them to one of several diets. They all receive instructions on how to follow their assigned diets and are followed-up on after one year. Turns out the Atkins people lost more weight than the other people. They also had a slightly more favorable lipid profile (i.e. higher HDL levels lower triglyceride levels) but elevated LDL levels compared to the other diets.5
A few things to note:
- I think it is hilarious that earlier in his post Taubes criticizes the Red Meat Consumption and Mortality study because he claims “they use questionnaires that are notoriously fallible” to collect dietary information.6 Meanwhile he praises this “A to Z” study for its design without mentioning that the authors use telephone-administered dietary recalls, which are not bad per se but chosen mainly for their efficiency rather than robustness.
- One could make an argument that simply giving people some instruction on the diets in the beginning and then coming back in a year to measure outcomes is not the best study design. For example, many nutrition labs in the Fred Hutch (including mine) will actually provide all of the meals to be eaten during the study free of charge. This accomplishes a couple things: 1) It increases compliance with the diet7 and 2) It allows the researchers to strictly control the calories, vitamins, fat content, etc. I don’t think the above study was a poor design, but there are stronger (albeit more costly and time-consuming) feeding designs one could use when studying diet.
- I was not surprised that the Atkins group lost more weight. There are actually good and interesting reasons for why low-carb diets are very effective in that area, but I won’t get into that now. What did surprise me at first was that the Atkins group had lower triglyceride levels. But then I remembered that weight loss in general will produce that effect, especially if subjects are overweight at baseline. If the other diets produced as much weight loss as Atkins I would expect a similar reduction of triglycerides.8
- This study uses exclusively disease-free, non-diabetic, non-pregnant or lactating (etc…9) overweight and obese premenopausal women, so to extrapolate these results and suggest that the general population would be healthier if they adopted this diet is certainly dubious.
- This is a diet study that examines carbohydrates and weight loss and lipid profiles after 12 months. The Red Meat Consumption and Mortality study in question deals with red meat and cancer, CVD, and other mortalities after 28 years. The former does not and cannot refute the latter. They are almost unrelated. You can eat meat three times a day and have the bulk of your calories come from carbohydrates. You can be a vegan that eats a ton of fat and protein but very few carbs. They are not mutually exclusive.
Now you may be thinking that this is a guy who simply demands the highest levels of evidence. Sure Taubes may miss out on a ton of great knowledge obtained from epi studies but the man takes a hard line on evidence. He will only accept findings from human randomized controlled trials and you can’t fault him for that.
Except that he doesn’t.
If you take a look at some of Taubes’s other posts you will see he makes other controversial claims that run contrary to mainstream science.10 In other posts he conveniently uses epi studies to bolster his narrative while impugning the RCTs that run contrary to his point.
Some examples of this are his posts on sodium. He cherry-picks cohort studies and case-control studies as evidence that the sodium-hypertension link is one big hoax. He even cites ecological and cross-sectional studies which are among the least substantial types of studies not just in epidemiology but in all of science (they would be blue or green-ish on the above pyramid), and he misrepresents their results to claim that sodium does not cause hypertension. Cross-sectional and ecological studies literally cannot show causation; they are not designed to do so.
Meanwhile, Taubes tries to downplay one of the strongest and most-lauded RCTs on the subject (The DASH trials) saying that blood pressure change was only “modestly lower.” If you call nearly a ten-point decrease in blood pressure by only manipulating sodium intake11 “modest” then sure. I suppose I can’t argue with such vague wording.* He also says that while the researchers measured blood pressure, they failed to measure other things such as lifespan. Big deal, right? Let’s just forget about that stupid study that completely contradicts me. He says basically the same thing about the Cochrane reviews that do not support his position on the subject. Sure the reviews state that cutting back on salt will lower blood pressure, but it doesn’t prove that cutting back will make you live longer so who cares, amiright? By the way, Cochrane reviews are at the very tip of the aforementioned evidence pyramid.
At least he doesn’t sink to the level of anecdotal evidence and personal testimony to prove a point. I take that back; he does:
All I knew was that I had played high school football in suburban Maryland, sweating profusely through double sessions in the swamplike 90-degree days of August. Without salt pills, I couldn’t make it through a two-hour practice; I couldn’t walk across the parking lot afterward without cramping.
You know what? Of all the evidence on the subject that story of Taubes in high school is the smoking gun we have all been waiting for, so I take it all back. You were right all along Taubes. Go collect your Nobel Prize. You have earned it, my friend.
Mr. Taubes takes the position that red meat does not contribute to cancer or CVD mortality. He took this position without any evidence from RCTs since, as he said, they have never been done.12 Moreover, he claims that any epidemiological evidence against his position is meaningless because the epidemiology field itself is meaningless.13 The only evidence Mr. Taubes is willing to accept are those of RCTs which are nearly impossible when it comes to things like diet and cancer mortality. So Mr. Taubes has set things up to where he cannot be proven wrong even if he is wrong.
You know what? I can do that, too! I contend that parachutes are not beneficial and life-saving when it comes to falling out of the sky. Prove me wrong. Oh and you can’t point to instances where groups of people have jumped out of an airplane and the parachute has slowed their velocity toward earth allowing a safe landing while those who had a malfunctioning parachute or no parachute at all suffered major trauma or death. Those would have to be either cohort or case-control studies and therefore meaningless. There have been zero RCTs studying the effect of parachutes and gravitational challenge.14 The basis for parachute use is purely observational, and its apparent efficacy could potentially be explained by a “healthy cohort” effect. My contention stands!
If Mr. Taubes had any intellectual consistency he would have to agree with that point. At best he would have to remain neutral on parachutes, since there are no RCTs to prove they do anything.
Some Additional Nit-Picky Stuff
Nutritionists and public health authorities have gone off the rails in their advice about what constitutes a healthy diet.
Wow, that’s painting with a pretty broad brush now isn’t it? It’s like saying “Government bureaucrats are wrong.” Much like government bureaucrats there are quite a few “nutritionists and public health authorities” in the world and they hold a variety of positions on a variety of topics. You mind picking one, telling me what it is, and why it is wrong? Or do you just want to construct a straw man that grossly misrepresents what a few people may or may not be saying so you can more easily refute it?
Are you talking about the advice of dietitians? What specific piece of advice do you object to? Do you have a problem with the following statement that I took off the AND website: “A well-balanced diet filled with whole grains, fruits, vegetables, healthy fats, low-fat dairy and lean protein is important for health and wellness.” Is that advice “off the rails”? Here’s a statement I copy-pasted from ChooseMyPlate.gov: “Limit the amount of foods and beverages with added sugars your kids eat and drink.” Is this bad advice? If so, why?
I first wrote about the pseudoscience of epidemiology in Science back in 1995, “Epidemiology Faces It’s Limits”[sic]… my Science article has since been cited by over 400 articles in the peer-reviewed medical literature…
Over 400 citations? Wow! Congratulations Mr. Taubes. You should be proud of yourself. Although I’m confused as to why you put that bit of self-aggrandizement in your post. Is it to imply that because your article has so many citations your thesis (Epidemiology is a bogus science) is correct? If citations = truth then I’m afraid that Willett (the villain in your narrative) is right and you are wrong. Why? Because Willett has actually published far more papers than you have, many of which have well over 1500 citations each. Furthermore, Willett’s papers are actual peer-reviewed scientific studies as opposed to simply lay commentary.
One last (petty) thing.
I’m writing this post with a little more haste than is my wont.
What are you doing Taubes? How about you quit pretending to be an 18th century British aristocrat and start acting like a 21st century American, you pretentious windbag. You are not Christopher Hitchens. Knock it off.
Way down at the bottom of the blog post Taubes admits a correction to an earlier version of the post. Evidently he instructed Dr. Willett to go read a chapter in a textbook titled Modern Epidemiology and learn how the “best epidemiologists” conduct real research. As it turns out Dr. Willett actually wrote the chapter in question. I won’t add any commentary here. Just let that bit of delicious schadenfreude wash over you like a warm bath.
*EDIT: This sentence originally said “more than ten-point decrease.” As it was pointed out to me by a commenter the more than 10 point decrease in blood pressure was between a low-sodium DASH diet compared to a typical high sodium diet. The data where ONLY sodium is manipulated resulted in decreases of 6-7 points.
- Pan A, PhD, Sun Q, MD, ScD, Bernstein AM, MD, ScD, et al. (2012) Red Meat Consumption and Mortality Results from 2 Prospective Cohort Studies. Arch Intern Med. 172(7):555-563.
- The number of studies published by Willett is currently at 1279 according to Pub Med
- WCRF/AICR (2007) Food, Nutrition, Physical Activity, and the Prevention of Cancer: A Global perspective. Second Expert Report. London, UK: World Cancer Research Fund; 280-288
- From the article: “The results were adjusted for age (continuous); body mass index (calculated as weight in kilograms divided by height in meters squared) category (<23.0, 23.0-24.9, 25.0-29.9, 30.0-34.9, or ≥35); alcohol consumption (0, 0.1-4.9, 5.0-29.9, ≥30.0 g/d in men; 0, 0.1-4.9, 5.0-14.9, or ≥15.0 g/d in women); physical activity level (<3.0, 3.0-8.9, 9.0-17.9, 18.0-26.9, or ≥27.0 hours of metabolic equivalent tasks per week); smoking status (never, past, or current [1-14, 15-24, or ≥25 cigarettes per day]); race (white or nonwhite); menopausal status and hormone use in women (premenopausal, postmenopausal never users, postmenopausal past users, or postmenopausal current users); family history of diabetes mellitus, myocardial infarction, or cancer; history of diabetes mellitus, hypertension, or hypercholesterolemia; and intakes of total energy, whole grains, fruits, and vegetables, all in quintiles.)”
- Gardner CD, Kiazand A, Alhassan S, et al. (2007) Comparison of the Atkins, Zone, Ornish, and LEARN Diets for Change in Weight and Related Risk Factors Among Overweight Premenopausal Women the A to Z Weight Loss Study: a Randomized Trial. JAMA. 297(9):969-977.
- By the way the food frequency questionnaires used in Red Meat Consumption and Mortality as well as many other studies certainly have their limitations. No one disputes that. But they have been repeatedly verified for validity and reproducibility in many other studies, especially when looking at overall dietary patterns.
- something that the authors of this study were concerned with by stating: “limitations included the lack of a valid and comparable assessment of individual adherence to the 4 different diets”
- In fact the authors mentioned exactly that: “[T]he trajectories of weight change between 6 and 12 months suggest that longer follow-up would likely have resulted in progressively diminished group differences.”
- “Women were excluded if they self-reported hypertension (except for those whose blood pressure was stable using antihypertension medications); type 1 or 2 diabetes mellitus; heart, renal, or liver disease; cancer or active neoplasms; hyperthyroidism unless treated and under control; any medication use known to affect weight/energy expenditure; alcohol intake of at least 3 drinks/d; or pregnancy, lactation, no menstrual period in the previous 12 months, or plans to become pregnant within the next year”
- Omigosh! I did not see that coming (/sarcasm)
- Sacks FM, Svetkey LP, Vollmer WM, et al. (2001) Effects on blood pressure of reduced dietary sodium and the Dietary Approaches to Stop Hypertension (DASH) diet. N Engl J Med. 344:3-10.
- and in this author’s estimation won’t be done in our lifetimes, if at all
- unless we’re talking about smoking, of course
- Smith GC, Pell JP. (2003) Parachute use to prevent death and major trauma related to gravitational challenge: systematic review of randomised controlled trials. BMJ. 20;327(7429):1459-61.