It’s a busy week, so I decided to put together some extra video footage from my interviews instead of writing a full post. This series of clips deals with sugar and starch, and how they’re pretty much the same.
We covered this territory in Fat Head pretty extensively, but not everyone who reads this blog has seen the film, and I believe it’s a message that bears repeating early and often — like voting in Chicago. My wife tells me she knows other mommies from her mommy group who won’t give their kids candy, but believe a lunch consisting of jam sandwiches with a side of fruit is just great — it’s lowfat, you see.
When I went through all the interview footage during editing, I of course had to choose from among many takes … I usually asked the same question a few different ways to make sure I’d have plenty of choices. So it’s kind of fun (for me at least) to hear the medical experts offering different explanations of the same ideas. Enjoy.
I’ll be spending a chunk of this week in Tennessee looking for a house, since it now looks as if we’ll finish up our house sale and hit the road within two weeks. I probably won’t have time to write another post this week, but I’ll check comments as often as I can.
Some time ago, I recommended checking out Jason Sandeman’s Well Done Chef! web site. Jason is a talented writer and a real chef who believes in real food. In addition to writing about food and issues related to food, Jason has quite a few recipes available on his site. Nothing like free lessons from a pro.
He recently offered to write an occasional guest column for the Fat Head blog, including instructions on how to prepare real food. I think it’s a great idea, because I sometimes receive emails from people asking for specific suggestions. I don’t have much to tell them, other than mentioning some low-carb cookbooks my wife uses.
Jason’s timing was also perfect (shouldn’t a chef have good timing?) because I usually write one of my bi-weekly posts on Thursday nights, and I’ll be away in Santa Barbara this Thursday. Drs. Mike and Mary Dan Eades are hosting a Fat Head night: a showing of the film for a group of their friends who haven’t seen it yet, followed by a Q & A with the writer/director. I’ll try to remember to snap some pictures.
So I’m stepping out, and Jason is stepping in. Here’s his guest column:
Curing Franken Stock! How to Make Your Own Chicken Stock
You might be thinking to yourself that you would like to make something out of that magazine you saw at the dentist’s office. Perhaps you need to make rice, and are reaching for that Uncle B’s Rice-A-trocious and have a gnawing sense inside you that all is not good. Perhaps you might even go as far as to look at the ingredients in the can:
Sounds just like what Grandma used to make, right? Here, a little information:
Autolyzed wheat: Basically dead yeast cells whose enzymes are broken down into easier to digest components. It is also known as a flavor enhancer akin to MSG.
Monosodium Glutamate: There have been so many reports railing against the evils of this additive. What most people do not know is that it naturally occurs in nature. For instance, seaweed has a lot. Does it belong in chicken broth? Only if you are trying to pass off water as something “wholesome and nutritious.” Modern MSG is manufactured through fermentation of sugar canes and starches.
Dextrose: Where else can you find corn byproducts in your food?
Hydrolyzed Wheat Gluten: This byproduct of wheat is used to give “body” to the product, disguising the thin, insipid nature of the broth. Nice!
Corn oil: Another by-product (way of getting rid of) of corn. Do we need it? Certainly not. We have to keep those farmers employed though, so we may as well put it in there.
Flavoring: What does this mean, anyway? Do we really want to know?
Hydrolyzed Corn Protein: One would say that this is a synonym for “MSG”. We definitely need more to make this taste anything like broth.
Hydrolyzed Soy Protein: Well, we have done everything else to fill the “need” for corn, so now we are onto soy! See, this by-product adds “texture” to the broth. Yum.
The reason this is available to the general public is that the manufacturers are doing us a favor. See, you do not know how to make the real stuff, so you must buy the canned stuff. I even get Kraft™ magazine which tells me to use the stuff.
I am here to change that for today.
There is good news about this and bad news. Let’s do the bad first, shall we? You will not be able to make this in 5 minutes. It is going to take a while, at least 4 hours.
That is not so bad though, is it? Set aside a Sunday morning to start this, and then we get into the good news: This is so easy, that an 8-year-old can do it, given the directions below. Once you start the process, you can let it happily simmer on the stove for the given time, and have great results. People will be flocking around your house to see what you are cooking that smells like Grandma’s did on a Sunday.
You will also have way more than a 10.5 oz (300 mL) can, and you can freeze it for when you need it next. You will be breaking free of Franken-Foods.
Let’s get started.
Chicken Stock (Broth)
Makes 1 gallon (4 liters)
2 chicken carcasses cut into 1 inch pieces
cold water to cover
1 onion, peeled, sliced into quarters
2 ribs celery, chopped coarsely
1 knot of ginger, halved
1/2 bulb garlic
4 bay leaves
1 bunch thyme
1. Wash chicken pieces in plenty of cold water; drain.
2. Place into pot and cover with cold water.
3. Place over medium-high heat and bring to the boiling point.
4. Reduce heat to a gentle simmer (the lowest your stove will go!) and leave to simmer for 4 hours.
5. Skim the top of foam and scum. (These are impurities, and you will not like them anyway!)
6. After three hours of gentle simmering, place onion, celery, ginger, garlic, bay leaves, thyme and peppercorns on the top.
7. Simmer for another hour, skimming when necessary.
8. Remove from heat and let the pot rest for 15 minutes so the bones can fall to the bottom.
9. Ladle the stock out through a fine strainer into a container. You may choose to use cheesecloth. Give yourself a star if this is the case.
10. Cool the stock completely.
11. Refrigerate and use within 4 days, otherwise pack it up, label it and freeze for up to 6 months.
After you have that, there is no reason to go back to the Franken Stock. Your dishes will automatically be better with them.
If you ever come across a recipe that calls for a can of stock, then you can use this interchangeably. You will notice the difference.
Last week I pointed out several flaws in how researchers gathered data for the NIH-AARP Diet and Health Study, which has generated a slew of scary headlines such as Animal Fat Linked to Pancreatic Cancer.
I also mentioned that even without those flaws, observational studies can at best only produce statistical associations. They don’t prove cause and effect … although you wouldn’t always know that from the headlines.
When people mention that obesity is associated with Type II diabetes and therefore must cause diabetes, I’ll sometimes reply that gray hair is also associated with diabetes and suggest we start giving Grecian Formula to everyone to prevent it. That usually generates a reply along the lines of, “Come on, that’s ridiculous. A lot of people develop diabetes when they’re older and happen to have gray hair.”
That’s the good news: people don’t confuse an association with a cause when it’s obviously ridiculous. The bad news is that if an association isn’t ridiculous, researchers often do believe they’re seeing cause and effect – especially if the association confirms a pre-existing bias.
Since observational studies produce so many alarmist headlines, I thought it would be a worthwhile exercise to recall just how spectacularly wrong a theory based on a statistical association can be. This is a real-world example that generated a lot of headlines back in the day.
For decades, heart-disease researchers have known what while women certainly do develop heart disease, they typically develop it later in life than men … usually after menopause. Naturally, this got the white-coat crowd wondering if female hormones – particularly estrogen – might protect against heart disease. The theory seemed to make sense: men don’t produce as much estrogen as women, and women don’t produce as much after menopause.
In the 1960s, men were given estrogen as part of a large clinical trial called the Coronary Drug Project – but that arm of the trial was stopped early because the men taking estrogen began dying from heart disease at a higher rate than men in the control group. So the theory was adjusted: estrogen appears to protect women from heart disease, but not men.
Then a major observational study gave the estrogen theory some real traction. For 15 years, the Harvard Nurses Health Study had been tracking the diets, health habits and disease rates of more than 120,000 nurses. When researchers pored over the mountains of data produced by that study, they found a startling statistic: women who took estrogen had a 40% lower rate of heart disease than women who didn’t. And women who continued taking estrogen were less likely to suffer a heart attack than women who took it for awhile and then stopped.
You can imagine the research papers and the headlines that resulted. There calls among researchers and doctors alike to start prescribing estrogen to all post-menopausal women who had risk factors for heart disease. More cautious researchers called for a controlled clinical trial before estrogen was given out like heart-healthy candy, and were criticized for it. How could they, in good conscience, deny this obvious wonder drug to millions of women while waiting for long clinical trials to play out?
A pharmaceutical company, Wyeth-Ayerst, eventually funded the clinical trials – hoping, of course, that estrogen would be shown to prevent heart disease. More than 16,000 women were randomized and enrolled in the study. For five years, half received estrogen and half received a placebo.
The results were hardly what Wyeth-Ayerst had expected: The women taking estrogen developed heart disease at a higher rate – 30% higher, in fact. They were also more likely to suffer a stroke … another cardiovascular disease. Later clinical trials confirmed the bad news.
The experts were flabbergasted. The statistical correlation in the Harvard Nurses Study couldn’t have been more convincing: women who took estrogen were far less likely to have a heart attack. And it couldn’t have been fluke – there were too many subjects involved.
So what happened? Nobody can say for sure, but some researchers at the time offered an explanation that makes perfect sense: the women in the Harvard study who took estrogen were more concerned about their health. That’s why they took a hormone replacement in the first place.
In other words, estrogen didn’t create healthy nurses, but health-conscious nurses did take estrogen. Meanwhile, the health-conscious nurses were less likely to develop heart disease … for any number of reasons.
This really isn’t all that surprising. In clinical trials, people who religiously take their pills tend to have better health outcomes than people who don’t. And guess what? It doesn’t matter if the pill they’re taking is the actual drug or the placebo. The difference is in the people, not necessarily in the pill.
Some people care about their health. Some people are lackadaisical about health. Researchers call them “adherers” and “non-adherers.” I have my own, more colorful labels. The point is, we’re talking about different kinds of people, and that difference can produce statistical correlations in observational studies that have little if anything to do with the true cause and effect.
Think about the estrogen studies again for a moment: we now know that estrogen doesn’t prevent heart disease and in fact can make it worse. And yet in a large, observational study, taking estrogen was associated with a steep reduction in heart disease – almost certainly because health-conscious women were more likely to take it.
Now think about some of the alarmist headlines and health-nanny propaganda you’ve read over the years, and ask yourself what’s really going on. Here a few examples I came up with:
Does a diet high in saturated fat cause cancer and heart disease? Nope. But since saturated fat has been demonized for 30 years, health-conscious people probably eat less of it.
Does giving up meat make you healthier? Nope. But most people who become vegetarians are probably health conscious.
Do whole grains prevent diabetes and cancer? Hell, no. But they’re less likely to cause those diseases than white-flour products, and health-conscious people are more likely to choose them.
Does watching Fat Head at least three times give you a high IQ? Uh … no. But I’d like to think there’s a strong statistical correlation.
The “Big Meal” news story that ran on ABC was, as Dr. Mike Eades explained quite thoroughly, a load of bologna passing for health and nutrition advice. I’d like to say that’s a rare occurrence. I’d also like to say I’m 30 years old, with 10% body fat, a full head of hair, and enough money in the bank to conduct a hostile takeover of Microsoft … but I’d be lying. Bad nutrition advice is everywhere.
I bookmark a lot of health and nutrition articles I find online, figuring I may write about them later. After the “Big Meal” story, I decided to go through them and pick out a few, separating them into the Good, the Bad, and the Ugly. I like ending on a positive note, so we’ll save the Good for last.
Cream-Style Corn: There was cream in your grandma’s recipe, but the liquid in today’s cans is actually a component of the corn kernels themselves and other ingredients such as starch. If you look at calories and fat, canned cream-style and whole-kernel corn are identical.
Ahhh, that explains why the obesity epidemic that ravaged the country in the 1950s is just a bad memory now. Silly ol’ Granny cooked with cream. (Can’t you just hear her yelling, “Jed! Go milk one of the cows so I can put some cream in yer vittles!”) Nowadays, being so much wiser, we cream up those starchy corn kernels by adding still more starch … kind of like they do at feed lots to provide us with well-marbled cows.
One large egg is a significant source of a number of vitamins and minerals, and contains only 75 calories and 5 grams of fat. Moreover, most of this fat is the healthy, unsaturated variety. (Eggs are high in cholesterol, but the chief villain in raising blood-cholesterol levels is not the cholesterol in our diets, but in saturated fats.)
Boy, they had me going for a minute, actually praising eggs. Then they had to go and point a finger at the chief villain. And by the way, that last sentence makes no sense. It’s not the cholesterol in our diets, but in saturated fats … ? Huh? If the cholesterol is hiding inside saturated fats (and probably wearing a fake moustache so we don’t catch on), how does it get into our bodies if not through our diets? Do we fall asleep and accidentally snort saturated fat particles floating around in the air?
Saturated fat doesn’t raise cholesterol. It raises HDL, which isn’t actually cholesterol, but a protein that carries old cholesterol from your tissues to your liver. High HDL is desirable. So snort that saturated fat out of the air and don’t feel guilty about it.
Chicken thighs: They are higher in fat and calories than breasts, but as long as you remove the skin and any excess fat, economical thighs fit into a good-for-you diet.
I sometimes eat chicken without the skin, but only because my daughter steals it when I’m going to the refrigerator for more butter. She loves the skin. Why? Because it’s fatty, and her little brain wants the fat. So does yours.
The Healthiest Carb You’re Not Eating. This writer heard about the wonders of whole grains at a conference. As she explains, “I was so blown away by the accumulating evidence of their protective effect against chronic disease that I made a beeline for the grocery store when I got home and purchased a box of the one brand of whole grain pasta I could find.”
In the studies that blew her all the way to the grocery store, researchers had one group eat white-flour products, while the second group consumed whole-grain products. The people who ate whole grains ended up with a lower rate of diabetes, among other ailments. So the researchers came to the obvious conclusion: whole grains prevent those diseases.
Anxious to repeat this rock-solid logic, I had one group of men smoke unfiltered cigarettes for 10 years, while a second group smoked the filtered variety. Turns out the men smoking the filtered cigarettes had a lower rate of lung cancer. This proves that filtered cigarettes prevent lung cancer. We should be passing them out in schools and using them to treat anyone who’s been exposed to asbestos.
Know this: It’s not the fat in foods that makes you feel full. That greasy cheeseburger will leave you feeling full all day, so it’s worth the splurge, right? Wrong! Fat is the slowest food component to clear the stomach, so for years it was assumed that fatty foods slowed digestion and kept you feeling full longer. Recent research proves the proportion of sugar and fat has little or no differences in satiety ratings. In fact, protein tends to leave people feeling more satisfied than either carbs or fat. The problem with fat is that it has more than twice the calories of protein or carbs.
That must be some interesting recent research, because a ton of old research demonstrates pretty clearly that given exactly the same number of calories, the higher the carbohydrate content, the hungrier people will be. Even the low-carb skeptics admit that people on low-carb diets end up eating less, which is why they call it “a low-calorie diet in disguise.” They just can’t seem to explain how the disguise curbs hunger. Perhaps hunger pangs are afraid of strangers wearing fake moustaches or other disguises, assuming they might be cholesterol preparing to hide themselves inside some saturated fats.
Know this: Low-carb diets don’t flush calories from the body. Proponents of low-carb diets claim you can excrete fat fragments (called ketones) in urine on this type of diet, essentially flushing calories out. But a study at Virginia Polytechnic Institute and State University in Blacksburg found no correlation between urinary ketone levels and weight change in women on low-carb diets.
Okay, they actually have a point. For all the good he did, I blame Dr. Atkins for the misconception that a positive reading for ketones in the urine automatically means you’re losing weight. It doesn’t; it simply mean you’re primarily burning fat for fuel. If I eat a big, fatty meal, I will get a positive ketone reading without losing weight, because my body has no need to tap my fat stores.
But here’s why ketones are good: Your brain prefers them for fuel, as do many other organs. And if you’re burning fat, it means your body isn’t relying on glucose, which in turn means your insulin levels are going to be lower. Then when you do need to tap your fat stores, they’re open for business. Otherwise, you can semi-starve yourself, and your body will respond by slowing your metabolism.
Low-fat Alfredo. Add two cups of plain soymilk (the thicker consistency mimics Alfredo) to a pot over low heat. Stir regularly to prevent sticking. Don’t bring milk to a boil. Add 1 tablespoon of dry milk powder, 1/2 cup of grated Parmesan cheese, salt, and pepper to mixture. Continue to stir until cheese is melted and milk is thickened. If mixture is not of the desired consistency, add dry milk powder and stir until dissolved. You’ll slash the fat calories by more than half!
Brilliant idea. While taking in a load of pasta to spike my blood sugar, I like to offset the effects by filling my body with soy and screwing up my thyroid. (I love alfredo sauce, so when I whip up a batch, I pour it over spaghetti squash or French-style green beans.)
Healthier alternative (to real salad dressing): Canned vegetable stock. Place in a pot on the stove and bring to a boil. Mix 1 tablespoon of cornstarch with 1/4 cup of cold water until dissolved. Add this mixture to the boiling liquid. This will thicken the stock and create the same consistency as high-fat, oil-based dressings.
Boy, doesn’t that sound delicious? Vegetable stock, with corn starch to thicken it up. This will also thicken your blood as your liver takes the starch and converts it to triglycerides. And by the way, without fat, your body won’t be able to absorb the fat-soluble nutrients contained any vegetables you put on your salad. So let’s see … none of the flavor, few of the nutrients, and a big load of triglyceride-raising starch. Pass me the Caesar dressing, now!
The fat-free craze didn’t begin in the ’90s though. For decades, scientists were warning of the dangers of fats. Saturated fat was deemed a killer, clogging the arteries and causing our pants to get tighter.
Never mind that for thousands of years indigenous cultures ate animal fat at nearly every meal and only in recent times had obesity become a problem and heart disease become a #1 killer.
It didn’t work. We are fatter and sicker than ever before. Many of us – myself included! – ended up with dry skin, constipation and a cold feeling in our bones. Our bodies’ need for fat manifested itself in magnificent cravings we satisfied at 3 a.m. with french fries and Ben & Jerry’s.
When I started eating more butter and taking fish oil, my skin really showed a difference. Lines in my face actually went away!
Bravo! I’m 50 years old, and I barely have any lines in my face. A year ago, I was carded while buying wine in a grocery store in Tennessee. When I asked the clerk if he was kidding, he said, “Sorry, we have to card everyone under 40.” I would’ve kissed him, but they frown on that kind of behavior in the south.
Now here’s the irony of the good, the bad, and the ugly: The bad and the ugly articles were provided by health magazines … Prevention, Men’s Health, Shape, etc. The good was featured in something called Women and Co., networking for the career-minded woman.
So we’ve got actual health nuts out there reading articles telling them to cut the fat and eat corn starch and pasta and soy milk. They’ll end up with arthritis, snap a ligament lifting weights at the gym, develop Type II diabetes while recovering in an easy chair and eating soy milk-smothered pasta, then wonder what the heck happened.
Meanwhile, career-minded women who want to network more effectively are learning they should be eating plenty of natural fats. They’ll be elbowing the health nuts out of their way as they power along to their next big meeting, looking fabulous in their business suits and their wrinkle-free skin.
I just hope the fat-eating career women network their way into taking over a few media empires. Then they can kick the health reporters’ tired, wrinkled butts.
A reader named Jane sent a link in a comment to a video of CSPI’s latest bologna-on-parade. Her link was to a Canadian TV story, the one below is to the ABC version:
Notice that ABC, like most media outlets, insists on referring to CSPI as a “consumer group.” No, they’re a bunch of vegetarians pushing a vegetarian agenda.
I replied to Jane that there’s so much wrong in this story, it deserves a full post. For one thing, you are much more likely to suffer a heart attack after a high-carb meal than after a high-fat meal. That’s not in dispute.
Well, it turns out Dr. Mike Eades already took this load of bologna and ground it up in his own post. He explained the science (and lack of it in this news story) better than I could, so I urge you to read his article. Here’s a sample:
It’s pretty impressive when the lab tech holds up the tube of blood taken after the meal and compares it to the one taken before the meal. There is a lot of fat swimming in the serum, that’s for sure. What the producers of this piece (and, sadly, the doctors commenting although they should know better) want you to take away from all this by the way they set it up is that all that saturated fat went directly into the blood. And how can you argue with them? It’s there for all to see.
Problem is, that’s what blood samples look like after almost any meal, especially one that contains carbohydrates. The fat you see isn’t the fat the two reporters ate; it is the fat the liver has made from the carbohydrate. It’s the same picture a tube of blood would show after either of the two doctors had eaten a high-carb, low-fat lunch.
If you find yourself scratching your head and wondering how so many people can still believe the Lipid Hypothesis despite all the evidence against it, now you know … medical reporting in much of our media is just plain awful.
Oh my gosh! I eat a lot of animal fat … I can feel my pancreas swelling up with tumors as I write. I’ve been issued a death sentence, and I know it’s accurate because – hold onto your seats, now – the article included the magic words study finds right there in the sub-headline.
Considering how many headlines this study has already produced – with more sure to follow – I’m going to suggest you memorize the name: The NIH-AARP Diet and Health Study. I’m also going to suggest that when you spot an article that cites this study, you bookmark it, download it, print it, and then use the pages to paper-train a puppy.
NIHARP (my shorthand) is one of those big, expensive studies that enables researchers to analyze data, publish research papers, give speeches, and otherwise pay their mortgages for years without ever seeking another grant. In fact, as the media likes to repeat over and over, this is THE LARGEST STUDY OF ITS KIND.
Wow, that must mean we’re looking at some rock-solid science here, right? Hardly.
Because NIHARP is typical in many ways of the studies that scared people away from fat, it’s worth taking a closer look. I downloaded quite a few study documents, including the original food survey, and I’ll try to explain the weaknesses of studies like this while keeping the statistical geek-speak to a bare minimum.
My girls have recently become huge fans of The Sound of Music. So, as the song says, let’s start from the very beginning …
Throughout 1995 and 1996, the investigators mailed a food-frequency questionnaire to 3.5 million members of the American Association of Retired Persons, all aged 50 to 69, who lived in six states (California, Florida, Pennsylvania, New Jersey, North Carolina, Louisiana), plus two metro areas (Detroit and Atlanta.) The authors said they chose these areas because they have high concentrations of retired people. I’m guessing that if people retired in California or Florida, it was for the weather, whereas if they retired in Detroit, they couldn’t afford to move.
Here’s the first big problem with the study (the largest of its kind!): the survey itself. In order to determine what people eat, the investigators sent them a list of 120 foods and asked them to answer questions like this:
Over the last 12 months, how often did you eat the following foods? (Ignore any recent changes.)
Whole milk (4%), NOT in coffee, NOT on cereal: Never | 1-6 per year | 7-11 per year | 1 per month | 2-3 per month | 1-2 per week | 3-4 per week | 5-6 per week | 1 per day | 2-3 per day | 4-5 per day | 6+ per day. Portion size: less than ½ cup | ½ to 1 cup | more than 1 cup.
Breads or dinner rolls, NOT INCLUDING ON SANDWICHES: Never | 1-6 per year | 7-11 per year | 1 per month | 2-3 per month | 1-2 per week | 3-4 per week | 5-6 per week | 1 per day | 2-3 per day | 4-5 per day | 6+ per day. Portion size: less than 1 slice or roll | 1 or 2 slices or rolls | more than 2 slices or rolls.
Mayonnaise or mayonnaise-like salad dressing on bread: Never | 1-6 per year | 7-11 per year | 1 per month | 2-3 per month | 1-2 per week | 3-4 per week | 5-6 per week | 1 per day | 2-3 per day | 4-5 per day | 6+ per day. Portion size: less than 1 teaspoon | 1 to 3 teaspoons | more than 3 teaspoons.
Ground beef in mixtures such as tacos, burritos, meatballs, casseroles, chili, meatloaf: Never | 1-6 per year | 7-11 per year | 1 per month | 2-3 per month | 1-2 per week | 3-4 per week | 5-6 per week | 1 per day | 2-3 per day | 4-5 per day | 6+ per day. Portion size: less than 3 ounces | 3 to 7 ounces | more than 7 ounces.
Could you answer a survey like that accurately? I couldn’t. In fact, I didn’t. When I was working for the National Safety Council, some genius in management decided everyone in the company should fill out a survey like this one. On a whole lot of the questions, I needed a box labeled “I have no freakin’ idea.” But there wasn’t one. So I did what all my pals at work did: I guessed.
And I was 25 years old, not 65. My memory was sharp then and it still is, but I couldn’t tell you what I ate last Tuesday, never mind last February. Of the nearly 3 million people who received the NIHARP survey but didn’t return it, how many do you suppose looked it and mumbled, “I have no freakin’ idea,” then tossed it in the trash?
But around 600,000 people did return the survey, which leads to the second problem: this is a self-selected group that doesn’t mirror the general population.
In the baseline data, it’s obvious that compared to the general population, the survey group is far more likely to be white (over 90 percent), well educated, and non-smoking. The authors admitted they were concerned about the low response rate (about 17 percent), but managed to discern that “a shifting and widening of the intake distributions among respondents compensated for the less-than-anticipated response rate.”
In other words, they declared this cross-section of the population varied enough for a study and decided to keep going. (Gotta pay that mortgage, you know.)
Here’s the third problem: the self-selected group was winnowed down even further by the investigators. Yes, it’s common practice to try to dump incomplete or suspicious data, but in explaining how they determined if a survey was sufficiently complete, they stated, “In calculating our initial cohort sample size of 350,000 we focused on a single nutrient, dietary fat.”
Hmmm … sounds to me like they already had an opinion about which nutrient would wind up being linked to cancer. If they could determine how much fat you ate, you were in. Why fat? Why not sugar, or white flour, or corn flakes?
Nearly ten years after the first survey, the authors mailed a similar questionnaire, along with others that asked about exercise, smoking and medications. Then they compared the respondents’ diets with their rates of various diseases, focusing primarily on cancer. That’s where they came up with all the crunchable numbers.
So how well do numbers like these crunch? That’s the fourth big problem: they don’t crunch very well. They’re more on the squishy side. In one of their many papers, here’s how the researchers evaluated the accuracy of their own food-intake data:
For the 26 nutrient constituents examined, estimated correlations with true intake (not energy-adjusted) ranged from 0.22 to 0.67 … When adjusted for reported energy intake, performance improved; estimated correlations with true intake ranged from 0.36 to 0.76.
Correlations of less than 0.1 are as good as garbage. The correlation shown, 0.9, is very strong. Correlations have to be this good before you can talk about accurately predicting the Y value from the X value.
If you want to think of it visually, a correlation of 1.0 gives you a perfect trendline: if smoking absolutely, positively causes lung cancer and is absolutely, positively dose-dependent, then you could plot the number of cigarettes smoked per day against the incidence of lung cancer, and you’d get one of those lines that starts at zero in the lower left and zooms straight to the upper-right corner.
Correlation examples (courtesy Tony at www.EmotionsForEngineers.com)
But for this study, the estimated correlation (after being adjusted upwards) is between 0.36 and 0.76. In other words, the investigators themselves estimate that the accuracy of their food survey is somewhere between lousy and decent. Well, decent might be stretching it. The same analysis of their own study included this statement:
However, previous biomarker-based studies suggest that, due to correlation of errors in FFQs and self-report reference instruments such as the 24HR, the correlations and attenuation factors observed in most calibration studies, including ours, tend to overestimate FFQ performance.
So the lousy-to-decent estimate might be overestimated. Kudos to them for saying as much. And yet from this data, they’re going to look for correlations between diets and diseases and write a slew of research papers on what they find.
Which brings us to the fifth big problem: the associations you find when looking at data depend largely on the associations you seek. In a study like this, you gather a huge amount of data, then you ask the data some questions. How you ask the question affects the answer.
Some months ago, the researchers asked this data if there was an association between red meat and colon cancer, and wouldn’t you know it, the data answered “yes.” At least that’s the story that made the headlines. But the truth is, the question they asked went more like this: “Do people who eat a lot of steaks, hot dogs, hamburgers, sausage, pizza, cold cuts, bacon and deli sandwiches have a higher rate of colon cancer?”
Grouping all those foods together under the label “red meat” confounds the question – and it wasn’t necessary to confound the question. In the food survey, “steaks” is a separate item. If you really want to know if red meat causes cancer, why not simply ask, “Do people who eat a lot steaks have a higher rate of colon cancer?” Maybe they did ask that question. Maybe they didn’t like the answer, so they asked it again and included pizza and hot dogs.
Here’s another strange grouping: the food survey lumped butter and margarine together as a single food item. I nearly jumped out of my skin when I read that one. Talk about confounding the data! Butter is natural. Margarine is a processed frankenfood. The only similarity is that people spread them on toast. You may as well lump cigarettes and carrot sticks together because they have the same shape.
Even when researchers ask well-designed questions, there’s the “don’t ask, don’t tell” problem: there may be associations lurking in the data that no one is looking for. When Ancel Keys cherry-picked six countries and went looking for an association between fat and heart disease, he found it. But the same overall data showed a much stronger association between sugar and heart disease … and an even stronger association between television ownership and heart disease.
Which brings us to the sixth problem: Associations are only useful for providing clues. They don’t identify the cause. There’s a strong association between obesity and type II diabetes. Does that mean being fat causes diabetes? Nope. It could mean diabetes makes you fat. Or, more likely, it could mean obesity and diabetes are both caused by excess insulin. You get the idea.
Considering that the animal fat will kill you! message has been around for more than 30 years, it’s highly likely that people who eat a lot of pizza, hot dogs, hamburgers, bacon, sausage and deli sandwiches are the “non-adhering” types Dr. Mike Eades wrote about awhile back. (Or, as I call them, “people who don’t give a @#$%.”)
Those same people may also consume more sugar, more white flour, more high-fructose corn syrup, more cough syrup, etc. – which is not much of a stretch, when you consider that pizza, hot dogs, hamburgers and deli sandwiches are all served with a load of starch. But as far as I can tell, the NIHARP investigators aren’t asking questions about sugar and starch. So far, they seem interested in discovering that animal fat is dangerous, while fruits and vegetables will save your life.
The next time you see yet another paper from this study (the largest of its kind!) generate yet another round of alarmist headlines about the possible dangers of animal fats (and you will), keep this in mind about The NIH-AARP Diet and Health Study:
What we’re looking at is 1) a survey study with a low response rate that 2) required old people to accurately recall what they’d eaten in the past year (twice), which then provided data that is 3) almost certainly polluted by self-selection and confounding variables, and is 4) being analyzed by researchers who indicated from the beginning that their main concern is dietary fat, all for the purpose of 5) identifying associations, which don’t tell us very much anyway.
Other than that, it’s a fine piece of work. Now go fry up some bacon, and don’t worry about your pancreas. But try to avoid throwing the pan out the window.
(Hat tip to Mike Eades for Twittering the video. I nearly did a spit-take with my coffee.)