The latest “Low-Carb Diets Are Deadly!” study, the topic of my previous post, was based on data extracted from about a dozen food questionnaires mailed out over a 20-year span. Because so many studies rely on these questionnaires to draw headline-grabbing conclusions about associations between diets and disease, it’s important to understand just how lousy the accuracy rate is.
Twenty-five years ago, I completed one of those questionnaires as part of a wellness program sponsored by the company that employed me. (In fact, we were all required to complete one.) Many of us ended up laughing at the stupidity of the thing. Imagine being a young, single male who eats at least half his meals outside the home. Now try to accurately answer this question:
How many servings of ground beef did you consume in the previous month?
The truthful answer would’ve been “I have no friggin’ idea.” But I was required to answer the question. So my method of guesstimation went something like this:
Let’s see … I think I ate lunch at McDonald’s three times last week … no, wait … was it four times? Did I have Thai food with Frank last Friday, or was that the week before? Ah, hell, let’s say four lunches at McDonald’s. Okay … I think maybe I had the chicken sandwich once. No, it was twice. No, it was once. So I’ll say I had burgers three times. Did I get the Double Quarter Pounder that one day? I think I skipped breakfast and ended up eating the double. So that’s … well, if one burger patty is a serving, I must’ve had four patties. No idea about the whole month, so I’ll multiply four times last week by four weeks in a month and say I had sixteen burger patties. Maybe Mexican food twice, I might’ve had beef and broccoli once … Okay, I’ll just round it to up 20 servings of beef for the month.
By the time I was halfway through the questionnaire, I was so annoyed by all the guesstimation required, I just started filling in numbers that sounded sort of believable. I know several co-workers did likewise, because we talked about it later.
Here’s another example posted on the Diabetes Update site:
I know first hand how inaccurate the standard food frequency questionnaire is because several years ago I was a subject in a long term study that used the standard nutritionist designed food intake questionnaire. During this time I was logging my actual food intake, with weighed portions, trying to understand my own pattern of weight loss, so I knew exactly what I was eating during any given day or month.
The nutritionist associated with the study emailed me the nutritional breakdown that I had supposedly eaten, based on my answers to the standard food frequency questionnaire. It bore no relationship at all to what I had eaten either in terms of calories or the percentages of my diet represented by protein, carbs, or fat. When I offered to send the study my actual food intake, I was told that the questionnaire they were using had been carefully validated and was standard in all nutritional studies and that there was no point in looking at what I had actually eaten.
Some questionnaires lump all red meats into a single category — in other words, a fresh steak and a highly-processed frozen beef lasagna dinner are counted the same — and some even put butter and margarine into the same category … natural butter fat, artificially processed corn oil, hey, makes no difference if they both spread nicely on your dinner roll.
From this kind of data, researchers at Harvard believe they know what people ate over a 20-year period. And since he likes their conclusions, Dean Ornish believes it too.