I recently came across yet another example of why observational studies tend to suck. You’ve probably seen the headlines, like this one from USA today: Men Who Can Do More Than 40 Push-Ups Far Less Likely To Develop Heart Disease.
Let’s take a look at the article:
Here’s one way to predict your heart health: get down and give me 41. A new study finds that men who can perform at least 40 push-ups in one attempt are much less likely to suffer from heart disease within the next 10 years.
Researchers from the Harvard T.H. Chan School of Public health say their report is the first to show how push-up capacity is linked to heart disease. They found that middle-aged men who can log more than 40 push-ups in a single try have a 96% reduced risk of developing the potentially deadly condition and other related ailments, such as heart failure, compared to those who can complete no more than 10 push-ups.
A 96% reduced risk?! Okay, guys, get on the floor and start doing push-ups! Having strong pecs and triceps obviously prevents heart disease!
I was of course suspicious when I saw this study came from the Harvard School of Public Health, which we ought to rename Meaningless Observational Studies R Us. Sure, it’s a good idea to stay in shape, and exercise no doubt protects against heart disease to some extent. But a 96% reduction based on the ability to do more push-ups? Something doesn’t smell right.
In a speech I gave many years ago, I highlighted the weakness of observational studies by pointing out that men who are bald are much more likely to suffer a heart attack than men sporting a full head of hair. If we applied Harvard School of Public Health logic, we would assume baldness somehow causes heart disease. Eventually, we’d end up with products like this:
But of course, the reason bald men have more heart attacks is that men lose their hair as they get older. They’re also more likely to suffer a heart attack as they get older. So baldness is “linked” to heart disease. I suspected the push-up study was based on similar nonsense. If we dig into the data, we might find the “link” exists because younger men can do more push-ups. I’m sorry to say I was right.
Here’s more about the study from USA Today:
For their study, the authors reviewed health data from 1,104 active male firefighters taken annually from 2000 to 2010. At the start of the study, the average participant was about 40 years old with an average body mass index of 28.7. The firefighters were tasked with performing as many push-ups as they could, and their treadmill tolerance was also tested.
By the end of the study period, 37 participants suffered from a heart disease-related condition — and 36 of those men weren’t able to log more than 40 push-ups in the initial test.
The average participant was 40 years old. Uh-huh … now let’s look at a table from the study giving us more detail on those participants:
Well, how about that? The average participant may have been 40 years old, but the mean age of the firefighters who could do more than 40 push-ups was 35. Among those who could do 10 or fewer push-ups, the mean age was 48. Big shock. Men lose their ability to do push-ups as they age. If we start with a group of 35-year-olds and another group of 48-year-olds, which group is going to suffer more heart attacks during the next 10 years?
But that’s only part of what makes this a meaningless study. One of the greatest risk factors for suffering a heart attack is smoking. Take a look at the figures I highlighted at the bottom of the chart. Among the men who could do more than 40 push-ups, just 6.7% were current smokers when the study began. Among those who could do 10 or fewer push-ups, 24% were current smokers. Or to use a related bit of data, among the more than 40 push-ups group, 69% were non-smokers when the study began. Among the 10 or fewer group, only 45.3% were non-smokers.
In the study itself, the authors of course state they applied regression models to account for age and BMI, blah-blah-blah. Interestingly, they don’t mention applying a regression model to account for smoking. Or perhaps they did, but chose not to mention the results.
In any case, just balancing for age and BMI presents a different picture:
Even after adjusting for age and BMI, we observed an independent association of push-up capacity with CVD outcomes. Increased capacity was associated with a lower risk for CVD outcomes, with the comparison of the 21- to 30-push-ups group vs the 0- to 10-push-up group being statistically significant (hazard ratio, 0.25; 95% CI 0.08-0.76), although the other group comparisons did not reach statistical significance.
Adjust for age and BMI (again, no mention of adjusting for smoking), and the only significant difference in push-ups vs. heart disease is between the 10 push-ups or fewer group and the 21 to 30 push-ups group. So much for that 96% reduction in heart attacks for men who can do more than 40 push-ups vs. men who can’t do more than 10.
This study doesn’t actually tell us diddly about the ability to do push-ups versus the likelihood of developing heart disease. It simply tells us that younger men and men who don’t smoke are less likely to develop heart disease during the next 10 years. And by the way, they can also do more push-ups. Duh. And yet in the media articles, we get quotes like this:
“Our findings provide evidence that push-up capacity could be an easy, no-cost method to help assess cardiovascular disease risk in almost any setting,” says the study’s first author, Justin Yang, an occupational medicine resident at the school.
No, your findings provide evidence that observational studies from Harvard are usually meaningless garbage dressed up as science. It’s a wonder anyone keeps funding these turkeys.
If you enjoy my posts, please consider a small donation to the Fat Head Kids GoFundMe campaign.
A couple years ago, it was if you could stand up from sitting on the floor with your legs crossed (used to be called “Indian style” before the PC police showed up) without using your hands, you would live a longer life. It’s always something.
I think it was Dr. Eades in one of his books (or maybe it was you!) talked about a surgeon who had done his own little side study while a “real” study was going on in his hospital. I don’t remember all the details, but it was something like they were studying prostate cancer or heart disease or something. He determined that men with facial hair were far less likely to succumb to whatever it was than those men who didn’t sport a mustache, beard, or goatee. That’s what the numbers showed. Must be true.
Wasn’t me, so it was probably Dr. Eades.
Well Bah. I can do 40 push ups. I am 65. Here I was thinking I was golden. JK I know it was hog wash too.
Based on the data, you need to do 41 to achieve the full protective benefit.
Well done Tom!
I think people who regularly eat push-ups have very, very incidence of heart disease.
I’m surprised they didn’t factor in income, find that higher income correlates to fewer pushups and more heart attacks, and recommend more study because it contradicts the wealthier-is-healthier paradigm.
They probably didn’t think of that one.
They pay people to do the push ups for them.
Some other strange elements :
– the 31-40 push-ups group has 50% more cardiovascular events than the 21-30 group, so if you can’t go up to 40, you’d better stay between 21 and 30.
– the groups don’t have the same size (from 75 in the 0-10 group, up to 389 in the 31-40 group)
– the worst: due to its small size, there is only one cardiovascular event in the >40 push up group. So with just one CV event more or less, you could conclude either that the reduced risk is only 48% instead of 96%, or that push-ups totaly prevents from heart attacks…
Yup, this is pretty much a worthless study.
This is some really great analysis, but doesn’t fully commit to changing the problem. Instead of just writing a blog – which is helpful to inform the general public – why not go over to JAMA Network Open and leave your comments on the actual article. I think that would start a conversation between the scientists who did the work and others reading the original study.
From “The Myth of Cardiovascular Health From Exercise.” based on Dr. Henry Solomon’s book “The Exercise Myth”. https://www.washingtonpost.com/archive/lifestyle/wellness/1985/01/16/the-exercise-myth-by-henry-a-solomon-md-harcourt-brace-jovanovich/f26dcbd6-f452-470d-a0e3-d65eb33ddcbe/
“The great news is that you will be convinced, after hearing this
information, that correct nutrition, NOT exercise, is the soundest
way to assure heart health.
The truth is that, at best, exercise slightly prolongs the time before a heart attack – it can’t and doesn’t prevent one.
We are all told constantly that exercise prevents heart disease. WRONG.“The evidence is unassailable – coronary heart disease develops and progresses during training. Exercisers die of heart disease despite exercise.”1
We are told that running to get the heart rate up is ideal exercise.
WRONG. “Running injuries are especially common [particularly the knees].If you weight 150 pounds then you endure over 100 tons of force per mile [through the knees, etc.].”2
We are told that everyone is becoming unhealthy due to inactivity.
WRONG. “…[Y]ou must be truly sedentary – a slug who sits or lies about all day long or barely crawls out of bed – to be at any risk from inactivity.”3
This is why house cats eating a proper carnivore diet maintain perfect
weight and perfect health despite little activity and often despite sleeping virtually all day.
We are told that exercise makes you live longer. WRONG again. “There
is no relationship at all between activity and lifespan – none.”4
We are told to take a “stress test” to assess our “cardiovascular fitness.”
WRONG – stress tests are inaccurate. “Stress tests are not sensitive enough, specific enough, or reproducible enough for anyone to be sure they are telling you anything correct.”5
More at this link.
http://brianpeskin.com/pdf/reports/ExerciseMyth-CAMB.pdf
I’m not opposed to exercise, of course. I exercise. But if exercise alone prevented heart attacks, Jim Fixx would still be alive.
Game show host Jack Barry was an avid runner. He, too, dropped dead of a heart attack while running.
Let’s not forget Philippides, who ran 25 mi to the battlefield near Marathon and back to Athens to announce the Greek victory over Persia in the Battle of Marathon — then dropped dead.
Great tradition, marathons. Running a race in honor of a man who died of a heart attack running the same distance.
If a study is badly designed, it does not mean its conclusions are wrong, it means you cannot conclude anything.
In this case we cannot conclude anything from the 40 push-ups test. Even if this test is eventualy proven wrong, it still won’t mean exercice is useless, it will just mean that an oversimplistic test won’t predict anything on something as complex as CVD.
The French National Institute of Health and Medical Research has just released a huge document reviewing the benefits of physical activity on several health conditions (CVD, diabetes, depression, asthma, cancer). The document is very well made and you can find the degree of proof (on a 3 level scale) on the parameters of each condition For example, the best degree of proof is on stroke relapse, it is not so good on coronary heart disease. The proofs are not very good on diabetes, but good on depression. Sometimes the benefit is just on the quality of life, not on the evolution of the disease (for cancer for example).
Nowhere in the study it is said you can cure or prevent any of these diseases if you run twice a week, or that a proper diet is useless.
Correct, you cannot conclude anything from this study. That’s why it’s annoying that Harvard trumpeted these results as if they’d discovered a good predictor of CVD. They did nothing of the sort.
The other issue is that there really is no way to adjust for these types of differences. How does one “adjust” for BMI? Sure, you can create some type of factor or function to “adjust” for BMI (or age or whatever), but it’s a mirage: you really have no idea how to adjust for this and anything you do is a guess. Also, this is a great way to introduce biases: of course higher BMI = worse. It has to be worse, right? Or maybe not:
http://www.todaysgeriatricmedicine.com/news/ex_092616.shtml
(Overweight = better for this study.) But since Harvard’s bias is that overweight = bad, I’m sure they adjusted to meet their biases.
Personally, as soon as I saw the headline, I ignored it. I knew it would be worthless.
I also suspect it’s in those “adjustment” routines that the temptation to torture the data emerges.
Tom has here performed the peer-review that this loo paper apparently didn’t get prior to its escape.
Strictly speaking, of course, it’s not peer review, since these “researchers” aren’t Tom’s peers, as they have no qualifications as professional humorists, even though that’s what their scribbling amounts to.
Credit to them for being inadvertently funny.
Harvard employs a bunch of rent seekers. Baldness & heart disease may be related via calcification… just a thought.
If baldness correlates strongly to heart disease within the same age group, that may be true.
Yesterday I heard about this on the radio. Out of curiosity I want to see how many I could do at 56 years old. I stopped at 45. Years ago I could only do 20. I felt so proud. I did it again this morning. Then I read Tom’s take. Sigh.
Keep doing the push-ups! Just don’t assume the ability to push-ups correlates to your risk of a heart attack.
That’s great, I can only do 11 and am the same age. I never was good at pushups, though. It’s in my genes.
I strongly agreed with Tom’s take on this paper — until I actually read it. Nowhere are the authors stating that increasing the number of push ups you do will reduce your chances of having CVD. The utility of the test that they endorse is that it is a cheap, but functional way to assess firefighters for their future chances of getting CVD. The comparison is to a much more expensive test, VO2-max.
And, yes, you might get a similar result from just using their age. But in most cases that isn’t going to fly legally.
BTW, that weird uptick in CVD for the next-to highest push up group is there for the next-to-highest VO2-max group. It also works against using age or BMI or smoking as predictive. What is that?
Nor did I claim the authors claimed doing push-ups prevents heart disease. But they did claim the number of push-ups you can do is a useful assessment of CVD risk, which clearly isn’t true. After adjusting for age, they found no significant differences comparing any group to any other group, except for a (barely) significant difference between the 10-or-fewer group and the >= 21-30 group.
If adjusting for age translates to no significant difference, that means if you have, say, a 45-year-old do push-ups, it tells you nothing about his heart health if he can do 20, 30, or 40 push-ups. The number of push-ups in this study was a function of age.
To employ an analogy: men tend to earn more as they get older. They also are more likely to suffer a heart attack as they get older. So annual income is a “predictor” of CVD. But asking a 45-year-old man what his income is tells you nothing about the health of his heart. A 45-year-old man earning $60,000 per year is no less likely to have a heart attack than a 45-year-old man earning $80,000 per year. The “predictor” is worthless.
What’s Wrong with Harvard—And necessity of reading beyond abstract etcetera.
Above I wanted to include the URL to SMBC on gocomics from yesterday.
https://www.gocomics.com/saturday-morning-breakfast-cereal/2019/02/21
i hope one one gets through. I also posted the URL to here there.
The test, whether it be age, push-ups, VO2-max, BMI, all correlated with CVD risk. The fact that adjusting for age removes statistical difference between all but 2 groups doesn’t prove that age is the causative factor behind CVD. In all likelihood age is responsible, but that doesn’t invalidate the push-up test as a predictor of future CVD. It just says it is probably no better than using age.
But for some purposes it may be illegal to use age as the test.
Right, but if it’s no better than age — and essentially just a marker for age — it’s a pointless study and a pointless measure.
It’s not pointless at all, it got someone some grant money and will probably spawn more studies requiring even more grant money. But apart from that it’s pointless.
Via the New York Times:
“I read about a vegan woman who recently got married and banned all meat eaters from her wedding. Then her friends and family were like, ‘Oh, no, we’re not allowed at your vegan wedding? Oh, bummer.’” — JIMMY FALLON
Heh-heh-heh …
Since watching “Science for Smart People” and both versions of “Wisdom of Crowds,” I seem to be looking for confounding variables in everything. One could say “rain is associated with thunder.” But does that mean rain causes thunder? No. Rain is also associated with lightning. Lightning causes thunder. Confounding variable right there. A rainstorm with no lightning has no thunder. Same thing in this study the people who could do more pushups were younger and healthier in general and thus less likely to develop heart disease. The ability to do more pushups was not the cause. It’s scary that these idiots are what we get out of Harvard these days.
Sorry to say we’ve been getting this kind of junk science from Harvard for a long time now.
The real risk is that some men (maybe even the firefighters) will try to do 41 pushups (to lower their risk, of course) and have a heart attack from overexertion.
I hope not. They might need to carry me out of a burning building someday.
I’ve always been able to stand up from crossed legs … Indian style.
Now that’s “cultural appropriation” I guess I should have a hernia and retire to bed.
Retire to a teepee and continue the appropriation.
My somewhat intuitive theory is that baldness is correlated to heart disease for a different reason besides age. It’s a stand in for Vitamin D deficiency. What better way for the body to create Vitamin D via sunlight than to leave the flattest area of the body exposed. Unfortunately, most of us then wear a hat or cap so they look younger.
Vitamin D deficiency, possibly also inflammation from bad oils. Boy, if I only I’d known this stuff in my 30s when I started losing my hair.
Is nobody else concerned that some firefighters can’t do 10 pushups?!
Let’s hope they’re driving the truck and not rushing into buildings to carry people out.
I cannot believe how many pages of drool about the value of being able to do 40 push ups I had to wade through before I found this article. It’s shocking how willing important sounding news outlets are to repeat whatever is trending.
Being able to do 40 push ups means that you can do 40 push ups. Linking that to heart health is absurd. There are many factors that contribute to heart health and many ways to measure heart health, and the random linking of firemen being able to perform x amount of push ups and their stress test results might mean…wait for it…that some firemen exercise more than other firemen and as a result are stronger and have greater endurance and are able to do a bunch of push ups. Being able to do a bunch of push ups is the result of specialization for that given exercise and has nothing to do with heart health. Nothing.