All week long, readers have been sending emails or posting comments asking about the latest study purporting to show a link between red meat and colon cancer. Here’s a sample headline from an online article about the study, followed by some quotes:
Red meat ‘increases risk of bowel cancer’
Eating less red meat could prevent 6000 cases of Australia’s second deadliest cancer every year, experts claim. They say studies confirm that consuming roast beef, lamb and pork increases your risk of developing bowel cancer.
The World Cancer Research Fund said there was convincing evidence of a link between bowel cancer and red meat in its latest report, which combines existing research with 10 new studies. It had already warned people to avoid eating processed meats and restrict their weekly intake of red meat to five medium portions – or 500g – to lessen the likelihood of developing bowel cancer.
Pretty scary so far, eh? Eating meat raises your risk of bowel cancer, so you’d better cut back on the stuff. Now check out this quote:
Milk appears to cut the risk of bowel cancer, but the WCRF does not recommend dairy foods because evidence for overall cancer risk is unclear.
Apparently the WCRF doesn’t view consistency as a necessary trait for researchers. Meat raises your risk of cancer, so you’d better cut back. Milk reduces your risk of cancer, but we’re not going recommend milk, because … uh … well … because the evidence on milk is unclear! Yeah, that’s how we get around that one!
The implication, of course, is that the evidence linking red meat to colon cancer is clear. So is it?
Hardly. In my Science For Smart People speech, I listed several critical-thinking questions we need to ask when some new study warns that this-or-that food is going to kill us. If you haven’t seen the speech, you may want to give it a look before continuing.
Let’s examine this latest study and ask ourselves those questions. (By the way, if you’re not as prone to banging your head on your desk as I am, you can download and read the full WCRF document.)
Q: Is this a controlled clinical study, or an observational study?
A: The WCRF study is a meta-analysis of other observational studies. In other words, they took the data from some observational studies and pooled the results. So in effect, this is yet another observational study. As I explained in the speech, observational studies at best provide evidence for a hypothesis. They don’t even come close to proving cause and effect. That alone means you could pretty much write off the sensational headlines and head out for a nice steak dinner. But what the heck, let’s continue.
Q: If A and B are linked, is possible that B causes A?
A: I’m going to step out on a limb here and conclude that developing colon cancer doesn’t cause people to eat more red meat.
Q: If A and B are linked, is it possible that A is connected to B because they’re both caused by C?
A: Dern tootin’ it’s possible. Health-conscious people — who are different in many ways from their I-don’t-give-a-hoot peers — avoid what they’re told is bad for them and gravitate towards what they’re told is good for them. Even if the WCRF researchers really and truly found a meaningful link between red meat and colon cancer (which they didn’t), it could simply mean that health-conscious people eat less red meat because they believe it’s bad for their health.
Q: Did the researchers control their variables?
A: Nope. Just as with a recent study I mentioned in my speech, the studies included in the meta-analysis lumped red meat and processed meat together. Here’s a quote from the full text:
The summary relative risk was 1.17 for colorectal cancer for each 100g/day increase in red and processed meats … The definition of red meat varied between the studies. Red meat was described by the studies as fresh red meat, red meat or a combined exposure of beef, pork and lamb.
In other words, we’re putting burgers, pizza, burritos, hog dogs, deli sandwiches, pigs in a blanket, prime ribs, pork chops, filets with béarnaise and racks of lamb with mint sauce all into one category. Whole foods or processed foods that are typically served with a heapin’ helping of white flour … all the same to us.
To be fair to the WCRF researchers, it would be nearly impossible to make the distinction anyway, since the data came from observational studies that were based on food-recall surveys. Which leads to another critical-thinking question I included in the original draft of my Science For Smart People speech, but deleted to stay within my allotted time:
Q: Is the data reliable?
A: HA-HA-HA-HA-HA-HA-HA-HA! Sure, about as reliable as a congressional budget forecast. The largest study included in this meta-analysis (and therefore given the most statistical weight) was the NIH-AARP Diet and Health Study, which I analyzed in a long post nearly two years ago. Here’s how the data for that study was gathered:
Throughout 1995 and 1996, the investigators mailed a food-frequency questionnaire to 3.5 million members of the American Association of Retired Persons, all aged 50 to 69, who lived in six states (California, Florida, Pennsylvania, New Jersey, North Carolina, Louisiana), plus two metro areas (Detroit and Atlanta).
Only 17% of those who received the surveys responded. They were overwhelmingly white, affluent and (of course) elderly. The investigators decided this wasn’t a problem because “a shifting and widening of the intake distributions among respondents compensated for the less-than-anticipated response rate.” Uh-huh.
Ten years later, similar food-recall surveys were sent out again, along with questions about any health and medical issues the respondents may have experienced. From these two data sets — gathered ten years apart — the researchers reached their conclusions.
Now, about those food surveys … have you ever taken one? I have, because I was required to by an employer. My co-workers and I found the questions impossible to answer accurately, so we just started making up the answers. Then we went out for happy hour (during which none of us recorded our food intakes). I’ve reproduced part of the NIH-AARP food questionnaire below:
Over the last 12 months, how often did you eat the following foods? (Ignore any recent changes.)
Whole milk (4%), NOT in coffee, NOT on cereal: Never | 1-6 per year | 7-11 per year | 1 per month | 2-3 per month | 1-2 per week | 3-4 per week | 5-6 per week | 1 per day | 2-3 per day | 4-5 per day | 6+ per day. Portion size: less than ½ cup | ½ to 1 cup | more than 1 cup.
Breads or dinner rolls, NOT INCLUDING ON SANDWICHES: Never | 1-6 per year | 7-11 per year | 1 per month | 2-3 per month | 1-2 per week | 3-4 per week | 5-6 per week | 1 per day | 2-3 per day | 4-5 per day | 6+ per day. Portion size: less than 1 slice or roll | 1 or 2 slices or rolls | more than 2 slices or rolls.
Ground beef in mixtures such as tacos, burritos, meatballs, casseroles, chili, meatloaf: Never | 1-6 per year | 7-11 per year | 1 per month | 2-3 per month | 1-2 per week | 3-4 per week | 5-6 per week | 1 per day | 2-3 per day | 4-5 per day | 6+ per day. Portion size: less than 3 ounces | 3 to 7 ounces | more than 7 ounces.
Could you answer those questions accurately about your diet for the entire previous year? No, I didn’t think so. But the WCRF researchers expect us to believe that based on questionnaires like this one, they can determine what people eat accurately enough to link various foods to minor changes in cancer rates. (Yes, minor changes … we’ll get to that soon.)
But let’s suppose for the sake of argument that we’re looking at accurate dietary information and move on …
Q: If a scientist proposes that A causes B, do we find that connection between A and B consistently and repeatedly, or are there glaring exceptions?
A: Allow me to quote the WCRF document. Here’s what they said about the first group of studies they analyzed:
Sixteen cohort studies on colorectal, or if not available colon cancer risk were identified. Eleven studies provided sufficient data to be included in the highest versus lowest analysis. Nine studies reported an increased risk for the highest versus lowest processed meat comparison, one was statistically significant. One study observed no association. In addition, positive associations were observed in men, and in women in the study that reported on the subgroups separately. Only the result in men was statistically significant.
Out of 11 studies with sufficient data, nine reported higher cancer rates for heavy meat-eaters … but only one was statistically significant, and only if they teased out the data for men separately. I guess women can eat all the red meat they want, but men will die of cancer if they chow down on burgers. The paragraph above also means, by the way, that two studies with “sufficient data” showed no increase in cancer rates among heavy meat-eaters.
Here’s the official word on the second group of studies analyzed:
Twelve cohort studies on colorectal cancer risk were identified and included in the highest versus lowest forest plot. Eight studies reported an increased risk for the highest versus lowest red meat comparison, one was borderline significant. Two studies reported no association and two studies observed a statistically non-significant inverse association.
Head. Bang. On. Desk.
Of the eight studies that showed an increase in colon cancer among heavier meat-eaters, apparently only one was “borderline significant” — it’s difficult to tell, since the sentence is poorly written. Two studies showed no increase, and two other showed a slight decrease in cancer rates among heavy meat-eaters. That means one-third of the studies — one-third! — failed to show a correlation between eating red meat (and processed meat) and colon cancer. If red (and processed) meat causes colon cancer, how can that possibly be? Were people in one-third of these studies somehow immune to red meat’s cancer-causing effects?
So there’s your evidence from the studies the researchers chose. Before we move on, here’s the conclusion from another meta-analysis of 14 studies published in 2004:
Greater intake of either red meat (excluding processed meat) or processed meat was not related to colorectal cancer risk.
So I’m guessing if we could somehow overcome cherry-picking and publication bias, even more than a third of the studies would fail to show a link between meat and colon cancer. A researcher who criticized a previous WCRF report certainly seemed to think there was some cherry-picking going on. Here’s part of what he wrote in the American Journal of Clinical Nutrition:
The report omits 13 cohort studies on red meat and colorectal cancer with a total of 1,578,970 subjects, including a very large 1992 study by the American Cancer Society and studies by Hirayama, Heilbrun et al, Goldbohm et al, Knekt et al, Gaard et al, Hsing et al, Jansen et al, Flood et al, Kojima et al, Chao et al, and Sato et al. All but 2 of these studies found no significant association with red meat.
The report omits the follow-up of 5 groups of vegetarians compared with socially matched omnivores by Key et al. They found no difference in mortality from colorectal cancer.
But back to the current WCRF analysis … Below, I’ve copied some charts that were included in the full text of the report. Take a look at this one:
In the chart above, you’re seeing the relative risk of developing colon cancer for each 100 grams of red (and processed) meat consumed each day. The black vertical line represents a relative risk of 1.0 – that is, a neutral risk. Notice that in two of the studies (Lee and Pietinen), each 100-gram increase in red (and processed) consumption was correlated with a lower risk of colon cancer. Now look at the last figure labeled Overall. That’s the statistical average for all the studies, and it’s the figure being trumpeted by the researchers.
Yup, based on some studies that showed a decrease in colon cancer among meat-eaters and other studies that showed an increase, the WCRF folks declared that we should all be cutting back on red (and processed) meats.
Now here’s the chart showing the dose-response change in relative risk for women:
Notice anything interesting there? Three of the studies showed cancer rates going down as meat consumption went up. The others showed a very slight increase in cancer rates. The average (for what it’s worth, which isn’t much) was essentially no change at all.
But wait, it gets even better. The chart below shows what happened to rates of colon cancer in several of the studies as meat consumption went up:
This is laughable … look at the studies by Lee, Wei and Jarvinen. Rates of colon cancer went up, then down again as meat consumption rose. I swear, next time I have lunch in a restaurant and see someone eating meat, I’ll feel obligated to offer a warning:
“Excuse me, but did you just eat some red (or processed) meat?”
“Why yes, I did eat some red (or processed) meat. So what?”
“Well, you better eat more red (or processed) meat for dinner, and lots of it. You don’t want to develop colon cancer.”
Rather than simply declare the data inconsistent — which it clearly is — the researchers plugged their pooled data into Excel, came up with a tiny average increase in colon cancer risk as meat consumption went up, then called a press conference. (Then probably sent out more fund-raising letters.) But again for the sake of argument, let’s assume that tiny average increase represents something meaningful and ask our next question:
Q: What was the actual difference?
A: The WCRF people proclaimed that the average relative risk of developing colon cancer among heavier meat-eaters (in the studies they chose to include) was 1.17, otherwise known as a 17% increase. Sounds kind of impressive, huh? A lot of media reporters certainly thought so. But what does that actually mean? In the Science For Smart People speech, I explained that the actual difference — otherwise known as absolute risk — means subtraction. Relative risk is calculated using division, which usually produces a more impressive (or more frightening) number.
According to some data I found online, the lifetime odds of developing colon cancer in America are 1 in 20. So with a little Excel magic, I extended the figures to a population large enough to avoid comparing fractions of people. Here’s how it would work out if eating red (and processed) meat really and truly produced a 17% increase in colon cancer:
In a population of 4,000 Americans, there would be about 100 lifetime cases of colon cancer among people with a lower intake of red meat, and about 117 lifetime cases of colon cancer among people with a higher intake of red meat. So the actual difference (117 – 100) would be 17 additional cases of colon cancer for every 4,000 people. I’m pretty sure I’ll beat those odds, especially since the relative risk was calculated using unreliable and inconsistent data.
Q: Do the results actually support the conclusions reached by the investigators?
Let’s tally up the score, shall we? The WCRF analysis is 1) based on observational studies that 2) lumped red meat and processed meat in the same category and 3) relied on food surveys that are notoriously inaccurate. The WCRF took the results of those observational studies, which 4) did not show a consistent link between red meat and colon cancer, and averaged the results in order to produce 5) a small increase in relative risk that translates to 6) a meaningless rise in absolute risk.
Based on that analysis, the researchers declared that there is now convincing evidence that red meat increases the risk of developing cancer.
Like I concluded in my speech: Scientists Are Freakin’ Liars.
One of the media articles I read about the study included this blaring headline:
Scientists confirm direct link between bowel cancer and red meat consumption
So I guess it’s time to reach another conclusion: Journalists Are Freakin’ Idiots.