In my last post, I compared the organizations that have been promoting artercloggingsaturatedfat! hysteria for the past 40 years to prosecutors who refuse to believe they put an innocent man in prison — even when new evidence says that’s exactly what happened. Never mind that pesky new DNA test, they insist. You have to look at the totality of the evidence. Someone who either works for or believes in the American Heart Association even left a comment to that effect (along with a couple of links):
You don’t take fringe studies from 50 years ago that contradict the vast majority of lipid research over the last half century and make a conclusion. You look at the body of evidence.
Well, the Sydney Diet Heart Study wasn’t exactly a “fringe” study. It was a controlled clinical trial that ran for seven years and included 458 subjects. Unless human biology has changed in the past 50 years, the results are still relevant.
I’ll examine that “vast majority of lipid research” the commenter linked to in a moment. First I’d like to deal with the argument that we must consider the totality of the evidence.
No, we don’t. Good scientists don’t consider a hypothesis to be validated unless the evidence supporting it is consistent and repeatable. As the science philosopher Karl Popper explained, if your hypothesis is that all swans are white, as soon as I start finding black swans, your hypothesis is wrong. It’s been falsified. If you find 100 white swans and I only find three black ones, you might insist that the “totality of the evidence” is in your favor, and it is – but your hypothesis is still wrong.
Another reason I don’t buy the “totality of the evidence” argument is that selection bias and publication bias are both rampant in nutrition science. I’ve read papers where the conclusions simply weren’t supported by the actual data. Studies that don’t produce the results the investigators wanted are often buried. If your academic paper supports conventional wisdom, it’s far more likely to be published. As Dr. Uffe Ravnskov can tell you from personal experience, papers that challenge conventional wisdom are often rejected over and over, with little or no explanation … unless you consider “this just has to be wrong” an explanation. So when researchers decide to do a meta-analysis of published studies, there’s a good chance they’re analyzing a stacked deck.
With that in mind, let’s start by looking at some of the “totality of the evidence” offered by the arterycloggingsaturatedfat! crowd, then move on to a few black swans.
The first link from our “body of evidence” commenter was to this study, a meta-analysis of eight studies. And how were those studies selected? Did the investigators go out and examine the entire body of evidence? Hardly. Here’s a quote from the study:
Of 346 identified articles, 290 were excluded based upon review of the title and abstract. Full texts of the remaining 54 manuscripts were independently assessed in duplicate by two investigators to determine inclusion/exclusion. Forty-six studies were excluded because they did not meet inclusion and exclusion criteria.
Most of the “body of evidence” was excluded merely by reading titles and abstracts. I’m not claiming the investigators rejected studies that didn’t support conventional wisdom, but the potential for cherry-picking is certainly there. Out of 346 studies they identified, they ran their analysis on just eight.
To their credit, the researchers discussed the weaknesses of the eight studies they selected:
Many of the identified randomized trials in our meta-analysis had important design limitations. For example, some trials provided all or most meals, increasing compliance but perhaps limiting generalizability to effects of dietary recommendations alone; whereas other trials relied only on dietary advice, increasing generalizability to dietary recommendations but likely underestimating efficacy due to noncompliance. Several of these trials were not double-blind, raising the possibility of differential classification of endpoints by the investigators that could overestimate benefits of the intervention. One trial used a cluster-randomization cross-over design that intervened on sites rather than individuals; and two trials used open enrollment that allowed participants to both drop-in and drop-out during the trial. The methods for estimating and reporting PUFA and SFA consumption in each trial varied, which could cause errors in our estimation of the quantitative benefit per %E replacement. One of the trials also provided, in addition to the main advice to consume soybean oil, sardines to the intervention group, so that observed benefits may be at least partly related to marine omega-3 PUFA rather than total PUFA consumption.
Enough said about that one.
The commenter also left a link to this study, another meta-analysis published by our pals at the American Heart Association. Here’s pretty much all you need to read to realize we’re not exactly looking at overwhelming evidence:
Several randomized trials have evaluated the effects of replacing saturated fatty acids with PUFAs on CHD events. Intakes of PUFA (almost entirely omega-6 PUFA) ranged from 11% to 21%. In addition to the inability to double-blind these studies, many had design limitations such as small sample size (n=54), the provision of only ≈50% of meals, outcomes composed largely of “soft” ECG end points, randomization of sites rather than individuals with open enrollment and high turnover of subjects, use of vegetable oils that also contained the plant omega-3 fatty acid α-linolenic acid, and simultaneous recommendations to increase fish and cod liver oil use.
Think we have enough confounding variables there? The “totality of the evidence” doesn’t appear to be very convincing. I believe an intelligent jury would have to conclude there’s more than a little reasonable doubt. So let’s move on to some black swans.
The American Heart Association’s advice is based on the belief that saturated fat raises cholesterol, and high cholesterol causes heart disease. Replace those arterycloggingsaturatedfats with polyunsaturated or monounsaturated fats, and you’ll lower your cholesterol, thus preventing a heart attack.
Does eating saturated fat raise your cholesterol? Perhaps, but not necessarily. Look what happened in this study:
Multiple food allergies required a group of seven patients with elevated serum cholesterol levels to follow a diet in which most of the calories came from beef fat. Their diets contained no sucrose, milk, or grains. They were given nutritional supplements. This is the only group of people in recent times to follow such a diet. During the study, the patients’ triglyceride levels decreased from an average of 113 mg/dl to an average of 74 mg/dl; at the same time, their serum cholesterol levels fell from an average of 263 mg/dl to an average of 189 mg/dl. At the beginning of the study, six of the patients had an average high-density lipoprotein percentage of 21%. At the end of the study, the average had risen to 32%. These findings raise an interesting question: are elevated serum cholesterol levels caused in part not by eating animal fat (an extremely “old food”), but by some factor in grains, sucrose, or milk (“new foods”) that interferes with cholesterol metabolism?
When I cut the carbohydrates and increased my animal-fat intake, my total cholesterol dropped, my LDL dropped, my triglycerides dropped, and my HDL went up. That’s what happened with the patients in this study as well. So perhaps saturated fat, if it’s harmful at all, is only harmful in the context of diet that includes sugars and grains – exactly the point Dr. Richard Feinman has made over and over.
Switching to vegetable oils can indeed lower cholesterol levels, but that does translate to fewer heart attacks? That’s what the American Heart Association tells us. But once again, I see black swans in the sky. In the Sydney Diet Heart Study that started this debate, the men who switched to safflower oil lowered their cholesterol by an average of 13%. That means a subject whose total cholesterol was 230 (deemed unsafe by the AHA) would, on average, end up with total cholesterol of 200 – the recommended level. And yet the men whose cholesterol was lowered by substituting polyunsaturated fats for animal fats ended up with a higher mortality rate, including higher mortality from heart disease. I’d call that a rather dramatic black swan.
In this study, the researchers were able to lower cholesterol levels by having a control group of men switch to soybean oil. Here are the results:
The test diet lowered the serum-cholesterol from a mean initial figure of 272 to 213 mg. per 100 ml. at six months (22% fall); the level in the controls fell from 273 to 259 mg. per 100 ml. (6% fall). Suspected relapses were assessed at regular intervals by a review committee unaware of the patient’s diet group. 62 men on the test diet suffered at least one relapse during the period of observation compared with 74 of the controls. 40 of the first relapses in the test group were major (i.e., definite reinfarctions or deaths from coronary heart-disease) compared with 39 major first relapses in the controls. The total number of men who had a major relapse at any time in the trial was 45 in the test group and 51 in the controls; of these major relapses 25 in each group were fatal. None of the differences found is significant. Relapses were not related to initial cholesterol level, to change in cholesterol level during the trial, nor, in any consistent way, to observance of the dietary regimen.
Once again, we see a dramatic fall in cholesterol levels, but no difference in heart-attack deaths. That looks like another black swan to me.
Here’s still another clinical study in which polyunsaturated fats replaced saturated fats:
Four hundred fifty-eight men with coronary heart disease participated in a trial of secondary prevention for 2 to 7 years. Overall five year survival was 81%. For those with first heart attacks it was 86%.
From infarct to entry into the trial the majority of men had already made changes in their diet and smoking habit and has lost weight. They were allocated randomly to two dietary groups. In one group the diet consumed derived 9.8% of calories from saturated fatty acids and 15.1% from polyunsaturates. In the second group saturated fatty acids contributed 13.5% and polyunsaturated fatty acids 8.9% of total calories. Survival was slightly better in the second group. Multivariate analysis showed that none of the dietary factors were significantly related to survival.
No significant difference, and what difference there was favored the group that consumed more saturated fat.
I saved my favorite for last. In this study, men with existing heart disease were divided into three groups: the control group maintained their normal diet, a second group was instructed to cut back on animal fats and consume polyunsaturated corn oil, and the third group was instructed to cut back on animal fats and consume monounsaturated olive oil. You know, olive oil – the stuff that’s supposed to prevent heart attacks. Here’s a quote from the full text of the study:
Patients in both oil groups were instructed to avoid fried foods, fatty meat, sausages, pastry, ice-cream, cheese, cakes (except plain sponge), etc. Milk, eggs, and butter were restricted. An oil supplement of 80 g./day was prescribed, to be taken in three equal doses at meal-times. The general nature and purpose of treatment were explained, together with the fact that different patients were receiving different kinds of oil. No advice on dietary fat was give to control patients.
The men in the two invention groups were monitored and counseled every month at first, then every two months. Investigators then tracked cardiac deaths and cardiac events for two years. The control group, by the way, was the oldest on average: 58.8 years vs. 55 years for the olive-oil group and 52.6 years for the corn-oil group. Considering that heart-disease rates increase dramatically as people move through their fifties, I think that’s significant. Nonetheless, here are the results from the study:
At two years the proportion of patients remaining free of major cardiac events is greater for the control group (75%) than for the two oil groups (olive oil 57%, corn oil 52%). The likelihood that the difference between the control and corn-oil groups was due to chance is 0.05-0.1 (S.E. of difference, + 13 %). Among those patients who were followed for a third year no new trend emerged.
Strangely, the researchers concluded that corn oil isn’t a good treatment for heart disease, but said nothing about olive oil.
That study was published in 1965. Here’s a bit of text from the American Heart Association’s current guidelines online:
Eat foods with monounsaturated fats and/or polyunsaturated fats instead of foods with high levels of saturated fats or trans fats. Check out our Fats and Sodium Explorer tool to get your personal daily calories and fat and sodium limits.
Choose: Vegetable oils and margarines with liquid vegetable oil as the first listed ingredient. Examples are canola, corn, olive, peanut, safflower, sesame, soybean and sunflower oils.
Corn oil and safflower oil — two oils that produced higher cardiovascular mortality rates in clinical studies.
I rest my case.
If you enjoy my posts, please consider a small donation to the Fat Head Kids GoFundMe campaign.
I enjoyed reading this while dipping my buttery ham in the yolk of a fried egg over easy. I hope it was OK that I washed it all down with a glass of raw milk.
It’s certainly okay by me.
I think you forgot to mention one other thing at the end of your exposé:
Head. Bang. On. Desk.
;0)
I’m going to a father-daughter dance with my daughters tonight and didn’t want to have forehead bruises.
And the beat goes on — the beat-up-on-low-carb, that is. A recent issue of a women’s magazine had a little blurb about a study comparing two diets — one a “low-carb” diet which had 300 calories for breakfast, and the other a “high protein, high carbohydrate” diet which had 600 calories for breakfast. The high carb breakfast included dessert foods (the illustration was a cupcake with sprinkles). The “high-protein, high carb” diet was reported to have won hands-down, and the people on the low carb diet to have gained back nearly twice as much after the diet. Was the low carb diet also hight fat? high protein? truly low carb? Who knows? We’re told nothing further about it. I guess I shouldn’t be surprised. The only weight-loss diet suggested in the editorial of the magazine was Weight Watchers.
That figures.
I enjoyed reading this while dipping my buttery ham in the yolk of a fried egg over easy. I hope it was OK that I washed it all down with a glass of raw milk.
It’s certainly okay by me.
I think you forgot to mention one other thing at the end of your exposé:
Head. Bang. On. Desk.
;0)
I’m going to a father-daughter dance with my daughters tonight and didn’t want to have forehead bruises.
If you look at “the totality of the evidence” for flying saucers, there is a lot of evidence, a lot of evidence. Indeed, so much weak evidence that a cynic might wonder why, with flying saucers flocking to earth in such numbers, there is not any strong evidence.
I’m pretty sure the people who claim to have seen flying saucers are only giving us observational evidence.
The commenter also seems to have forgotten that the (well-published) “fringe study” (probably one of the few decent n6 studies ever) included a complimentary metastudy. He should take a look at the results – they are… Interesting.
Indeed.
If you look at “the totality of the evidence” for flying saucers, there is a lot of evidence, a lot of evidence. Indeed, so much weak evidence that a cynic might wonder why, with flying saucers flocking to earth in such numbers, there is not any strong evidence.
I’m pretty sure the people who claim to have seen flying saucers are only giving us observational evidence.
“With the cutoff number at the lower end of the normal range (200 mg/dl), they could include all of the healthy normal citizens in the range that would need treatment with diet, and since the diet would never work to permanently lower those normal levels (eg, 200 mg/dl to 240 or 260 mg/dl) to below 200 mg/dl, they could recommend that all these people should go onto cholesterol-lowering medications.”
But this is exactly what I did, i.e lower my T/C from 236 in July 2011 to 220 in January 2012 to 193 in June 2012, which by the way is the same as it was when I was 41 (am 54 now). Of course, I didn’t (and don’t) follow “the diet”. My LDL dropped from 131 to 112, and my HDL went down a little less, from 97 to 85, Trigs down from 54 (January 2012) to 41.
The commenter also seems to have forgotten that the (well-published) “fringe study” (probably one of the few decent n6 studies ever) included a complimentary metastudy. He should take a look at the results – they are… Interesting.
Indeed.
I’m currently reading Bad Pharma by Ben Goldacre..he also wrote a book called Bad Science all about this sort of thing. Every time I pick it up to read more, I terrify myself, but it is an awesome book. Unfortunately I will never trust a doctor again…
Consider that a healthy skepticism.
“With the cutoff number at the lower end of the normal range (200 mg/dl), they could include all of the healthy normal citizens in the range that would need treatment with diet, and since the diet would never work to permanently lower those normal levels (eg, 200 mg/dl to 240 or 260 mg/dl) to below 200 mg/dl, they could recommend that all these people should go onto cholesterol-lowering medications.”
But this is exactly what I did, i.e lower my T/C from 236 in July 2011 to 220 in January 2012 to 193 in June 2012, which by the way is the same as it was when I was 41 (am 54 now). Of course, I didn’t (and don’t) follow “the diet”. My LDL dropped from 131 to 112, and my HDL went down a little less, from 97 to 85, Trigs down from 54 (January 2012) to 41.
I’m currently reading Bad Pharma by Ben Goldacre..he also wrote a book called Bad Science all about this sort of thing. Every time I pick it up to read more, I terrify myself, but it is an awesome book. Unfortunately I will never trust a doctor again…
Consider that a healthy skepticism.
Some hope that the message is getting out there. From Fox News, an article that finally gets it right:
http://www.foxnews.com/health/2013/02/15/great-cholesterol-myth/?intcmp=features
Some hope that the message is getting out there. From Fox News, an article that finally gets it right:
http://www.foxnews.com/health/2013/02/15/great-cholesterol-myth/?intcmp=features
@Bex: It’s been my impression that one of the worst things to befall the field of medicine was the approval of direct-to-consumer advertising. It has blown the money-making/marketing aspect/political control of medicines all out of proportion, crippling the actual art and practice of doctors helping people get/be well.
@Bex: It’s been my impression that one of the worst things to befall the field of medicine was the approval of direct-to-consumer advertising. It has blown the money-making/marketing aspect/political control of medicines all out of proportion, crippling the actual art and practice of doctors helping people get/be well.
Again it is so much fun to read the comments on your site. I hope you didn’t mind but I’ve scraped that scatterplot you did on the MONICA study data into my FB page challenging people to find a correlation. Of course I also posted the link to the Jimmy and Monica article. I’d be happy if I manage to be a vehicle to drive more traffic through your blog.
I don’t mind at all.
Again it is so much fun to read the comments on your site. I hope you didn’t mind but I’ve scraped that scatterplot you did on the MONICA study data into my FB page challenging people to find a correlation. Of course I also posted the link to the Jimmy and Monica article. I’d be happy if I manage to be a vehicle to drive more traffic through your blog.
I don’t mind at all.
Brilliant, Tom, brilliant! So much bandwagon riding, selective “evidence”, looking to find supprt for preconceived ideas, it goes on and on and on.
Australia’s National Health and Medical Reshearch Council’s just published Guidelines are hysterically funny; over 80% of the recommendations are based on class C evidence, “Suggestive Association”: “**** MAY have positive outcome on xxxxx condition! e.g salt reduction “Reducing sodium intake by about 1,000 mg/day is associated with reduced risk of cardiovascular events.” The class A evidence is :”Decreasing consumption of sodium decreases blood pressure in normotensive adults; a reduction of 1,800 mg reduces systolic blood pressure by about 2 mmHg and diastolic blood pressure by about 1 mmHg.” Gawd, what a mind-blowing lowering of bloodpressure. The rest of the guidelines follow such monumental discoveries and related advice.
Yup, despite the evidence that even drastic reductions in sodium intake barely budges blood pressure, they insist on pushing the same lousy advice.
Brilliant, Tom, brilliant! So much bandwagon riding, selective “evidence”, looking to find supprt for preconceived ideas, it goes on and on and on.
Australia’s National Health and Medical Reshearch Council’s just published Guidelines are hysterically funny; over 80% of the recommendations are based on class C evidence, “Suggestive Association”: “**** MAY have positive outcome on xxxxx condition! e.g salt reduction “Reducing sodium intake by about 1,000 mg/day is associated with reduced risk of cardiovascular events.” The class A evidence is :”Decreasing consumption of sodium decreases blood pressure in normotensive adults; a reduction of 1,800 mg reduces systolic blood pressure by about 2 mmHg and diastolic blood pressure by about 1 mmHg.” Gawd, what a mind-blowing lowering of bloodpressure. The rest of the guidelines follow such monumental discoveries and related advice.
Yup, despite the evidence that even drastic reductions in sodium intake barely budges blood pressure, they insist on pushing the same lousy advice.