Studies The American Heart Association Doesn’t Want You To Read

In my last post, I compared the organizations that have been promoting artercloggingsaturatedfat! hysteria for the past 40 years to prosecutors who refuse to believe they put an innocent man in prison — even when new evidence says that’s exactly what happened.  Never mind that pesky new DNA test, they insist. You have to look at the totality of the evidence. Someone who either works for or believes in the American Heart Association even left a comment to that effect (along with a couple of links):

You don’t take fringe studies from 50 years ago that contradict the vast majority of lipid research over the last half century and make a conclusion. You look at the body of evidence.

Well, the Sydney Diet Heart Study wasn’t exactly a “fringe” study.  It was a controlled clinical trial that ran for seven years and included 458 subjects.  Unless human biology has changed in the past 50 years, the results are still relevant.

I’ll examine that “vast majority of lipid research” the commenter linked to in a moment.  First I’d like to deal with the argument that we must consider the totality of the evidence.

No, we don’t.  Good scientists don’t consider a hypothesis to be validated unless the evidence supporting it is consistent and repeatable.  As the science philosopher Karl Popper explained, if your hypothesis is that all swans are white, as soon as I start finding black swans, your hypothesis is wrong.  It’s been falsified.  If you find 100 white swans and I only find three black ones, you might insist that the “totality of the evidence” is in your favor, and it is – but your hypothesis is still wrong.

Another reason I don’t buy the “totality of the evidence” argument is that selection bias and publication bias are both rampant in nutrition science.  I’ve read papers where the conclusions simply weren’t supported by the actual data.  Studies that don’t produce the results the investigators wanted are often buried.   If your academic paper supports conventional wisdom, it’s far more likely to be published.  As Dr. Uffe Ravnskov can tell you from personal experience, papers that challenge conventional wisdom are often rejected over and over, with little or no explanation … unless you consider “this just has to be wrong” an explanation.  So when researchers decide to do a meta-analysis of published studies, there’s a good chance they’re analyzing a stacked deck.

With that in mind, let’s start by looking at some of the “totality of the evidence” offered by the arterycloggingsaturatedfat! crowd, then move on to a few black swans.

The first link from our “body of evidence” commenter was to this study, a meta-analysis of eight studies.  And how were those studies selected?  Did the investigators go out and examine the entire body of evidence?  Hardly.  Here’s a quote from the study:

Of 346 identified articles, 290 were excluded based upon review of the title and abstract. Full texts of the remaining 54 manuscripts were independently assessed in duplicate by two investigators to determine inclusion/exclusion. Forty-six studies were excluded because they did not meet inclusion and exclusion criteria.

Most of the “body of evidence” was excluded merely by reading titles and abstracts.  I’m not claiming the investigators rejected studies that didn’t support conventional wisdom, but the potential for cherry-picking is certainly there.  Out of 346 studies they identified, they ran their analysis on just eight.

To their credit, the researchers discussed the weaknesses of the eight studies they selected:

Many of the identified randomized trials in our meta-analysis had important design limitations.  For example, some trials provided all or most meals, increasing compliance but perhaps limiting generalizability to effects of dietary recommendations alone; whereas other trials relied only on dietary advice, increasing generalizability to dietary recommendations but likely underestimating efficacy due to noncompliance. Several of these trials were not double-blind, raising the possibility of differential classification of endpoints by the investigators that could overestimate benefits of the intervention. One trial used a cluster-randomization cross-over design that intervened on sites rather than individuals; and two trials used open enrollment that allowed participants to both drop-in and drop-out during the trial. The methods for estimating and reporting PUFA and SFA consumption in each trial varied, which could cause errors in our estimation of the quantitative benefit per %E replacement. One of the trials also provided, in addition to the main advice to consume soybean oil, sardines to the intervention group, so that observed benefits may be at least partly related to marine omega-3 PUFA rather than total PUFA consumption.

Enough said about that one.

The commenter also left a link to this study, another meta-analysis published by our pals at the American Heart Association.   Here’s pretty much all you need to read to realize we’re not exactly looking at overwhelming evidence:

Several randomized trials have evaluated the effects of replacing saturated fatty acids with PUFAs on CHD events.  Intakes of PUFA (almost entirely omega-6 PUFA) ranged from 11% to 21%. In addition to the inability to double-blind these studies, many had design limitations such as small sample size (n=54), the provision of only ≈50% of meals, outcomes composed largely of “soft” ECG end points, randomization of sites rather than individuals with open enrollment and high turnover of subjects, use of vegetable oils that also contained the plant omega-3 fatty acid α-linolenic acid, and simultaneous recommendations to increase fish and cod liver oil use.

Think we have enough confounding variables there?  The “totality of the evidence” doesn’t appear to be very convincing.  I believe an intelligent jury would have to conclude there’s more than a little reasonable doubt.  So let’s move on to some black swans.

The American Heart Association’s advice is based on the belief that saturated fat raises cholesterol, and high cholesterol causes heart disease.   Replace those arterycloggingsaturatedfats with polyunsaturated or monounsaturated fats, and you’ll lower your cholesterol, thus preventing a heart attack.

Does eating saturated fat raise your cholesterol?  Perhaps, but not necessarily.  Look what happened in this study:

Multiple food allergies required a group of seven patients with elevated serum cholesterol levels to follow a diet in which most of the calories came from beef fat. Their diets contained no sucrose, milk, or grains. They were given nutritional supplements. This is the only group of people in recent times to follow such a diet. During the study, the patients’ triglyceride levels decreased from an average of 113 mg/dl to an average of 74 mg/dl; at the same time, their serum cholesterol levels fell from an average of 263 mg/dl to an average of 189 mg/dl. At the beginning of the study, six of the patients had an average high-density lipoprotein percentage of 21%. At the end of the study, the average had risen to 32%. These findings raise an interesting question: are elevated serum cholesterol levels caused in part not by eating animal fat (an extremely “old food”), but by some factor in grains, sucrose, or milk (“new foods”) that interferes with cholesterol metabolism?

When I cut the carbohydrates and increased my animal-fat intake, my total cholesterol dropped, my LDL dropped, my triglycerides dropped, and my HDL went up.  That’s what happened with the patients in this study as well.  So perhaps saturated fat, if it’s harmful at all, is only harmful in the context of diet that includes sugars and grains – exactly the point Dr. Richard Feinman has made over and over.

Switching to vegetable oils can indeed lower cholesterol levels, but that does translate to fewer heart attacks?  That’s what the American Heart Association tells us.  But once again, I see black swans in the sky.  In the Sydney Diet Heart Study that started this debate, the men who switched to safflower oil lowered their cholesterol by an average of 13%.  That means a subject whose total cholesterol was 230 (deemed unsafe by the AHA) would, on average, end up with total cholesterol of 200 – the recommended level.  And yet the men whose cholesterol was lowered by substituting polyunsaturated fats for animal fats ended up with a higher mortality rate, including higher mortality from heart disease.  I’d call that a rather dramatic black swan.

In this study, the researchers were able to lower cholesterol levels by having a control group of men switch to soybean oil.  Here are the results:

The test diet lowered the serum-cholesterol from a mean initial figure of 272 to 213 mg. per 100 ml. at six months (22% fall); the level in the controls fell from 273 to 259 mg. per 100 ml. (6% fall). Suspected relapses were assessed at regular intervals by a review committee unaware of the patient’s diet group. 62 men on the test diet suffered at least one relapse during the period of observation compared with 74 of the controls.  40 of the first relapses in the test group were major (i.e., definite reinfarctions or deaths from coronary heart-disease) compared with 39 major first relapses in the controls. The total number of men who had a major relapse at any time in the trial was 45 in the test group and 51 in the controls; of these major relapses 25 in each group were fatal. None of the differences found is significant. Relapses were not related to initial cholesterol level, to change in cholesterol level during the trial, nor, in any consistent way, to observance of the dietary regimen.

Once again, we see a dramatic fall in cholesterol levels, but no difference in heart-attack deaths.  That looks like another black swan to me.

Here’s still another clinical study in which polyunsaturated fats replaced saturated fats:

Four hundred fifty-eight men with coronary heart disease participated in a trial of secondary prevention for 2 to 7 years. Overall five year survival was 81%. For those with first heart attacks it was 86%.

From infarct to entry into the trial the majority of men had already made changes in their diet and smoking habit and has lost weight. They were allocated randomly to two dietary groups. In one group the diet consumed derived 9.8% of calories from saturated fatty acids and 15.1% from polyunsaturates. In the second group saturated fatty acids contributed 13.5% and polyunsaturated fatty acids 8.9% of total calories. Survival was slightly better in the second group. Multivariate analysis showed that none of the dietary factors were significantly related to survival.

No significant difference, and what difference there was favored the group that consumed more saturated fat.

I saved my favorite for last.  In this study, men with existing heart disease were divided into three groups:  the control group maintained their normal diet, a second group was instructed to cut back on animal fats and consume polyunsaturated corn oil, and the third group was instructed to cut back on animal fats and consume monounsaturated olive oil.  You know, olive oil – the stuff that’s supposed to prevent heart attacks.  Here’s a quote from the full text of the study:

Patients in both oil groups were instructed to avoid fried foods, fatty meat, sausages, pastry, ice-cream, cheese, cakes (except plain sponge), etc. Milk, eggs, and butter were restricted. An oil supplement of 80 g./day was prescribed, to be taken in three equal doses at meal-times. The general nature and purpose of treatment were explained, together with the fact that different patients were receiving different kinds of oil. No advice on dietary fat was give to control patients.

The men in the two invention groups were monitored and counseled every month at first, then every two months.  Investigators then tracked cardiac deaths and cardiac events for two years.  The control group, by the way, was the oldest on average:  58.8 years vs. 55 years for the olive-oil group and 52.6 years for the corn-oil group.  Considering that heart-disease rates increase dramatically as people move through their fifties, I think that’s significant.  Nonetheless, here are the results from the study:

At two years the proportion of patients remaining free of major cardiac events is greater for the control group (75%) than for the two oil groups (olive oil 57%, corn oil 52%). The likelihood that the difference between the control and corn-oil groups was due to chance is 0.05-0.1 (S.E. of difference, + 13 %). Among those patients who were followed for a third year no new trend emerged.

Strangely, the researchers concluded that corn oil isn’t a good treatment for heart disease, but said nothing about olive oil.

That study was published in 1965.  Here’s a bit of text from the American Heart Association’s current guidelines online:

Eat foods with monounsaturated fats and/or polyunsaturated fats instead of foods with high levels of saturated fats or trans fats.  Check out our Fats and Sodium Explorer tool to get your personal daily calories and fat and sodium limits.

Choose:  Vegetable oils and margarines with liquid vegetable oil as the first listed ingredient.  Examples are canola, corn, olive, peanut, safflower, sesame, soybean and sunflower oils.

Corn oil and safflower oil — two oils that produced higher cardiovascular mortality rates in clinical studies.

I rest my case.