Archive for the “Random Musings” Category
In comments for my previous post about regression to the mean, someone asked a perfectly logical question: yes, a study subject’s cholesterol might be spiking on the day he’s screened, which means he’s enrolled in study of a cholesterol-lowering drug even though he doesn’t actually have high cholesterol. But doesn’t it work in reverse as well? Wouldn’t that occurrence of random chance be offset by someone whose cholesterol happened to be lower than usual on screening day?
The short answer is: no, the two random variations don’t offset. Let’s assume the cutoff for being enrolled in a study of a cholesterol-lowering drug is 230. If the guy whose true average cholesterol is 215 happens to spike at 235 on screening day, he’ll be enrolled. Meanwhile, the guy whose true average is 245 but happens to have a cholesterol level of 225 on screening day won’t be enrolled.
For the long answer, let’s show randomness in action with a larger group — say, 100 people. It would be easy to choose cholesterol numbers to demonstrate my point, but that would be cheating. So we’ll fire up Excel and put it to work.
First, we’ll create a subject pool. From what I’ve read, the average cholesterol level among adults living in industrial societies is around 220 – that is, if they’re not on statins. (The average is lower in many countries now because almost everyone with “high” cholesterol is given a prescription.) So we’ll create a pool of 100 potential subjects whose cholesterol levels more or less mimic the real-world population.
To do that, I told Excel to give 10 of the subjects a random cholesterol number between 140 and 300, the next 20 subjects a random number between 170 and 270, another 20 subjects a random number between 190 and 250, and the final 50 subjects a random number between 205 and 235.
The result was a wide range of cholesterol numbers with some outliers, but with the majority clustering within a stone’s throw of 220. I called those numbers the Real TC. Then to toss random chance into the mix, I told Excel to adjust each number by a random number between -30 and 30. I called those numbers the Screen TC. That’s the total-cholesterol number the researchers would record as the baseline for each patient.
As you can see, the averages of both the Real TC and the Screen TC (the one adjusted by random chance) are very close to each other, and also very close to the average among non-medicated adults. So to a researcher, these screening numbers would look about right.
But they’re not. Thanks to our random variations, some people whose true average cholesterol is above 230 will have a screening number below 230, and some people whose true average cholesterol is below 230 will have a screening number above 230. In theory, the effect would be the same in both directions. Therefore it’s no problem, right?
Wrong!!! It’s a huge problem. The true high-cholesterol subjects who score below 230 solely because of random variations are excluded from the study. In the data shown above, that would happen with Patient #97.
Meanwhile, the true low-cholesterol subjects who score above 230 solely because of random variations are included in the study. In the data above, that would be the case with Patient #99. The end result is that people whose true cholesterol is lower than their screened cholesterol are overrepresented in the patient pool. Once again, let’s let the numbers tell the story.
Remember, only people with a total cholesterol of 230 or above on screening day are enrolled in the study. So among those enrolled, there can be three effects of the random variation: someone with true high cholesterol can be screened as having even higher cholesterol (H > H), someone with true high cholesterol can be screened as having lower cholesterol that’s still at or above the cutoff of 230 (H > L), or someone with true low cholesterol can be screened as having high cholesterol (L > H).
After our random screening (courtesy of Excel), 35 of the 100 people screened ended up in the study. There were 11 subjects with true high cholesterol whose screening number was even higher, eight with true high cholesterol whose screening number was lower, and 16 with true low cholesterol (below 230), but who were screened at 230 or above — and thus ended up in the study group. That means out of our 35 study subjects, 27 have true cholesterol that’s lower than the number assigned on screening day. And again, this is all due to nothing more than random variations interacting with a cutoff number for enrollment.
Are you with me so far? Good.
Now let’s assume regression to the mean kicks in. At the end of the study, people whose cholesterol was spiking on screening day return to their true, lower average, which we’ll call Final TC. Likewise, people whose cholesterol was artificially low on screening day return to their true, higher average – but there are fewer of them for the reasons I explained above. Here’s what our average numbers look like.
Between screening day and the final day, our group showed better than an 11-point drop in cholesterol on average. Woo-hoo! The drug works!
Uh, but wait … I didn’t treat these people with any drug. The effect is totally due to random variations spread evenly across the potential-subject pool. Yes, the variation was equal in both directions. But when the variation was sufficiently downward, someone with true high cholesterol was excluded from the study. When the variation was sufficiently upward, someone with true low cholesterol was enrolled. When everyone regressed to the mean, the statistical effect was a drop in average cholesterol (at least as measured) among those enrolled.
Okay, let’s add one more twist. If the subjects enrolled were split evenly into a treatment group and a placebo group, then everything I described above should apply equally to both groups. We might see an artificial drop in cholesterol numbers across both groups, but no real difference between groups. So the researchers would have to report that the drug was no better than a placebo.
But as Chris Masterjohn pointed out in the article I linked in my previous post, that assumes we’re talking about studies with sufficiently large patient populations that are properly randomized. Let’s see what happens if we split our 35 enrollees into small groups.
To make sure I’m not cherry-picking, I took our study group and had Excel assign each subject a random number from 1 to 4. Then I divided the subjects by those numbers.
Just looking at groups 1 and 2 shows what can happen with small study groups. Suppose group 1 had been assigned the placebo and group 2 had been assigned the cholesterol-lowering drug. Look at the numbers. (Again, the Final TC here is just a return to each subject’s true average.)
Wow! The placebo group’s total cholesterol dropped just six points on average. But the treatment group’s total cholesterol dropped by an average of 19 points! Woo-hoo! The drug works!
But once again, I didn’t treat anyone. These numbers are all being produced by nothing more than random variations followed by a regression to the mean.
Now, if you’re a clever sort, you may already be typing your next comment … something along the lines of Oh, yeah, Mister Smarty Pants? Who says everyone would regress to the mean? If there are random variations on screening day, wouldn’t there also be random variations on the final testing day? Huh? HUH?!! And wouldn’t those cancel each other out?
Okay, let’s see for ourselves. Since our drug didn’t actually do anything (because we don’t actually have a drug), I took the true average cholesterol for each patient and, once again, told Excel to apply a random variation of between -30 and 30. That number became our new Final TC.
Adding random variation to the final measurement did, in fact, reduce the dramatic difference between groups 1 and 2.
Now it’s 12 points lower vs. 19 points lower. Not so impressive. But let’s suppose Group 3 had been our placebo group and Group 2 was still our treatment group.
Once again, we see a drop of just six points in the placebo group, but a drop of 19 points in the treatment group. Woo-hoo! If we make like a pharmaceutical marketer and express the difference in relative terms, our treatment was better than 200% more effective than the placebo!
But once again, there was no actual treatment effect whatsoever. The difference is entirely random. I didn’t have to throw out any outliers I don’t like or cherry-pick any data, either. I had Excel do all the random variations and group assignments for me to ensure that I couldn’t cherry-pick. All I did was choose a cutoff number for the study – total cholesterol of 230 or higher – and let randomness do the rest.
And yet, largely because of the small study size, I found an impressive difference between two of my groups – even after applying random variations to the final cholesterol measurement.
To repeat a quote from Chris Masterjohn:
And thus we see that many published research findings are false. Some of these false findings exist because we would inevitably expect by the laws of probability for a small handful of well conducted, thoroughly reported, and appropriately interpreted studies to uncover apparent truths that are really false simply by random chance. This emphasizes the need to look at the totality of the data. Some will be false because of regression to the mean. This emphasizes the need to critically evaluate the data in each study.
Makes you hope your doctor has an understanding of statistics and a desire to dig into the research before prescribing that wonder drug. But I wouldn’t bet on it.
12 Comments »
Sometimes that darned working-for-a-living thing gets in the way of more important stuff — like writing blog posts.
I was too swamped with work on Monday to write a post, and I still am. Chareva’s visiting her parents, so I’m also on chickens-dogs-cat-kids duty for the next few days.
See you next week … I hope.
5 Comments »
Pop quiz: what’s the difference between the two x-y graphs below?
The answer is that they’re same graph — but the first picture above, we’ve zoomed in on one side of the bell curve.
In the book How Not To Be Wrong: A Guide To Mathematical Thinking, author Jordan Ellenberg points out that if you focus on a small part of a curve, it looks rather a lot like a line. Human like lines because they’re easy to understand. Plot some data and draw a line, and you have visual evidence that more of this produces more of that, or that more of this results in less of that. If the x-axis represents a span of time, you can even predict what the data will look like years from now by extending the line into the future.
The trouble is, real life tends to be more curved and less tidy than our beloved trendlines would have us believe. Two variables that were happily holding hands and running uphill together on our chart can suddenly break up and go their separate ways. So the trendline flattens out or reverses direction completely. That’s often how real life is, and if we don’t want to be wrong (as the title of the book says), we shouldn’t assume that a trendline heading in a particular direction will continue in that direction forever.
For example, you’ve probably seen headlines announcing that by the year 2040, everyone in America will be obese. Yup, every single person. Those predictions are based on charts that look something like this:
Researchers take a 25-year trendline that runs from 1990 to 2015 and simply extend it another 25 years. That is, of course, a wee bit ridiculous. Some decent-sized fraction of the population is highly resistant to becoming fat. Even if the Standard American Diet doesn’t undergo a positive shift (which I think it already is), the likely scenario is that when the actual data is plotted in 2040, it will look something like this:
Trendlines can flatten out. It happens all the time. It’s what we see when the law of diminishing returns kicks in. If I’ve been lifting weights once per month and switch to twice per month, I’ll likely become stronger. Kick it up to once per week and I’ll probably become stronger again. But at some point, more frequent workouts won’t make me stronger. There’s a limit to how quickly a body can produce new, stronger muscle cells.
Trendlines can also rise and then fall, producing a bell curve. More of this means more of that up to a point, but then even more of this leads to less of that. Interestingly (at least for those of us who enjoy reading about economics), Ellenberg introduces the concept by writing about taxes and the revenues they produce.
I’ve had debates with big-government-lovin’ acquaintances who are convinced there’s no national problem we couldn’t solve if we just had the political courage to seriously jack up taxes – especially on those darned rich people. In their minds, the relationship between tax rates and tax revenues looks like this:
Higher tax rates always produce higher revenues and therefore more government goodies for all – no matter how high the rates go.
But that’s not how it works in real life. Ellenberg doesn’t argue in favor of any particular tax rate, but he points out what any sane economist knows: at some point, higher tax rates produce less revenue, not more, because (among other reasons):
- People have less disposable income to buy goods and services, which means fewer people will be employed to provide them.
- More people decide to participate in the underground economy to avoid high taxes.
- Fewer people can save the capital to start a new business.
- People who already have the capital to start a new business decide they won’t bother if they’re taking all the risks but Uncle Sam is going to take most of the reward.
- People in the highest income brackets often quit working for the year once any additional income they earn is taxed at a rate they find unacceptable.
So the relationship between tax rates and tax revenues is a bell curve:
Again, Ellenberg doesn’t advocate for a particular tax rate. He simply points out that in any given set of economic circumstances, there’s going to be a rate far greater than 0% and far less than 100% that produces the most revenue. If we only focus on the left side of the curve, we’ll end up believing that higher rates always produce more revenue. If we only focus on the right side of the curve, we’ll end up believing that lower rates always produce more revenue. Both beliefs are wrong – and of course, the title of the book is How Not To Be Wrong.
One way to avoid being wrong is to understand that in real life, the relationship between this-and-that often takes the form of a bell curve. Starting from a low level, more of something may be good … but that doesn’t necessarily mean a LOT more is better. Starting at a high level, less of something might also be good … but that doesn’t mean zero is better.
We see the bell-curve relationship all the time in the health sciences. How much vitamin D should you get? The answer would surely look something like this:
Starting from a low level, more is better. But that doesn’t necessarily mean a LOT more is better. At some point, a LOT more is toxic.
How much protein should you eat? We know too little is bad for your health. You’ll lose muscle mass and your immune system will weaken. But too much can cause diarrhea and dehydration. So the relationship between protein and health is a bell curve.
But my bell curve probably doesn’t peak at the same point yours does. And a muscular athlete who engages in heavy workouts would need protein than I do. So the relationship between protein and health may look something like this:
Not every relationship in health is a bell curve, of course. If we’re talking about rat poison and health, the relationship probably looks like this:
Less is better, period. Sometimes that’s the reality. Chareva’s mother is highly allergic to walnuts. So for her at least, the walnut-to-health relationship would look very much like the rat poison-to-health relationship above.
But getting back to the How Not To Be Wrong theme, here’s a graphic representation of a belief I once held, but now consider wrong:
Metabolic health is the highest at close to zero carbs, then goes down from there. I believed that because I went from a high-carb diet to a low-carb diet, and my health dramatically improved. So for awhile, I figured if low is good, close to zero must be even better. It may be true for some people, but it’s clearly not true for everyone.
On the low-carb cruise, I was pleased to hear Dr. Justin Marchiagiani respond to a question about ketogenic diets by saying he doesn’t recommend them for everyone. He works with each patient to find his or her ideal carbohydrate intake, which will depend on a number of factors. For some, it’s 50 grams per day or less; others feel better and lose weight more easily at 100 grams per day or more. He said he feels at his best at between 75 and 100 grams per day. That’s about where I am now as well.
There is no level of carb intake that’s ideal for everyone. So the relationship between carbs and metabolic health looks something like this:
If we only focus on the right side of the curve, we’ll end up believing that more carbs always means worse metabolic health, so the ideal level of carb intake is close to zero.
But that ignores the left side of the curve. And that’s an easy way to be wrong.
45 Comments »
During the low-carb cruise, I interviewed Dr. Ann Childers about how diet affects mood and mental health. She’s a psychiatrist who works with children and has seen a real-food diet work wonders, so I wanted to get her on camera for the upcoming book and DVD companion. One clip I can pretty much guarantee will end up in the DVD is her describing when a teacher called to ask what new wonder drug she’d prescribed to a student previously diagnosed with behavior problems.
“Bacon and eggs,” Dr. Childers answered.
“Yes, but WHAT ELSE?” demanded the teacher.
Dr. Childers also mentioned something Dr. Weston A. Price observed during his travels around the world: people eating their traditional diets weren’t just physically healthier; they were mentally healthier too. Dr. Price noted many times how cheerful and optimistic these people were, and how quickly they rebounded from life’s setbacks.
I thought about that during our return trip home from the cruise, because it was the kind of day that could easily have produced a case of acute crankipantus extremitus in kids, but didn’t in ours.
We booked the cruise closer to the deadline than we should have. When we searched for return flights on Orbitz, our options were 1) a long day of travel or 2) an extra $200 per person for a short day of travel. We elected to save the $800 and endure the long day.
How long? Well, let’s see … we left the ship around 9:00 AM and were sitting inside the Ft. Lauderdale airport shortly after 10:00 AM. Our flight didn’t leave until 3:45 PM — and that flight was to Detroit to change planes. Three hours on that flight, then a three-hour-plus layover in the Detroit airport, then an hour-and-a-half flight to Nashville. Then wait for the luggage. Then catch a shuttle to long-term parking. Then make a half-hour drive to Franklin. By the time we walked into our house, we’d been traveling for 16 hours.
And here’s what surprised me, although perhaps it shouldn’t have: the girls never got into a funk or whined about anything. They made a wisecrack or two, asking me if I couldn’t have found a longer and more roundabout way to get home, but they were laughing about it, not whining. (I told them I’d signed us up for the scenic route.)
They read, they played games on their Kindles, they commented on the view outside the airplane’s windows, they watched some of the in-flight TV offerings, they talked to us and to each other. They laughed many times throughout the long day. When the shuttle bus let us out in the long-term parking lot at the Nashville airport, Sara broke into a little musical ditty she’d written to memorize our row number. They were still cheerful when we finally pulled into our driveway.
They’re the daughters of two people who don’t much like whiners, so sure, heredity and upbringing both figure into how they handled themselves. But I believe diet figured into it as well. The long trip home was after a week of eating quality (mostly) food. During the cruise, they had bacon, sausage, fruit and eggs for breakfast – no pancakes, cereal, waffles or glasses of juice. Lots of meats, seafood and vegetables for lunches and dinners. They even ordered escargots in garlic butter several times for an appetizer.
Other than the couple of times we let them have sugar-free cookies as an indulgence, they were eating make-your-brain-happy foods all week. During our three-hour layover in the Detroit airport, we had dinner at a Texas Longhorn steakhouse. Then we sat for another two hours, waiting to board the plane to Nashville – again, with nobody complaining or getting cranky.
Now for the flipside …
Sunday was, as I’m sure you’re aware, Father’s Day. On Saturday, I went out in the 90-plus heat and high humidity and spent four hours mowing the back pastures. I was so soaked with perspiration, during one of my cooling-off breaks, Alana asked if I’d dumped a bucket of water over my head.
Hard work? Yup, especially in that heat. But after a shower and a change of clothes, I was re-energized and ready to go walk a few miles around the nearby Westhaven neighborhood, which sponsors an annual music festival called Porch Fest. (The bands play on porches. Nearly every house in Westhaven has a big front porch.)
Afterwards, we walked back to the Mexican restaurant in Westhaven for our Saturday dinner. That’s my “carb nite” meal most weeks. I eat the rice and beans that come with my fajitas, plus a few corn chips. It’s high-carb, but no wheat. I wake up Sunday mornings feeling no ill effects.
Yesterday morning was no exception. When Chareva asked what I wanted to do for Father’s Day, I replied that I wanted us to clean out the garage, sweep, and put away all the tools we’d let pile up during our big Spring Project. (Isn’t that every dad’s dream on Father’s Day?) So we did. It was 90-something and humid again, but my energy level was good.
After I showered and the girls gave me their home-made Father’s Day cards, I decided it was enough of a special occasion to head out for an indulgence meal. I put it up for a vote, and the consensus was that we’d go to Mellow Mushroom in downtown Franklin for pizzas. I haven’t had pizza since my birthday in November and probably won’t again until my next birthday, so I thought it was a fine idea.
As I often say, if you’re going to eat something you know is bad for you, at least choose a meal that’s worth it. The pizzas at Mellow Mushroom are excellent, and therefore worth it — assuming we’re talking about a very occasional indulgence, that is.
I was reminded today why I only eat wheat a couple of times per year. I slept nearly nine hours last night, but I’ve been low-energy all day. I don’t feel depressed – that would be stretching it – but I can safely call it a case of the blahs. I drank three big mugs of coffee over the course of the morning but never felt totally awake.
Often after dinner, I run out to play a quick 18 holes of disc golf as the sun dips behind the tall trees across the highway from our property. Today the idea didn’t appeal to me. Nothing requiring energy or exertion appealed to me. If anything, I felt like taking an afternoon nap, although I didn’t because I had programming work to do.
In a previous post, I described how I considered myself a low-energy person back in my college days. I was also a regular wheat-eater in college. I felt today like I felt back then. Not exactly bad, but not good either. I certainly wouldn’t describe my mood today as optimistic, and if you’d told me to go push a lawnmower up and down a steep pasture for several hours in the heat and humidity, I can promise the reply wouldn’t have been cheerful.
The difference between today and my college days is that today’s low-energy feeling is temporary. I know the cause and the cure.
Good food, good mood. Not-so-good food, not-so-good mood.
Food equals mood.
57 Comments »
Our cat’s official name is Rascal, but we usually refer to him as Little Man. I gave him that name at some point after his successful campaign to convert me into a person who likes cats – at least one cat, anyway.
I’m pretty sure after he joined the family two years ago, he evaluated us all and figured out I was the only one who wasn’t delighted by his presence. Okay, he said to himself, I’ll work on him. He took to jumping on my lap when I was watching TV late at night – which scared me out of my skin the first few times – and settling down for a long purr.
Later, he decided to make me his sparring partner. Whenever he gets the chance, he jumps onto my office chair and adopts a fighting pose he probably imagines is intimidating. If I walk near the chair, he swipes at me, and the sparring is on. I try to poke him in various places, while he swipes at my hand and tries to catch a finger in his teeth. If he does catch a finger, he gives it an oh-so-gentle nip to let me know he won the round. I call this game of his En Garde, Mister!
(Little Man playing En Garde, Mister! with my hand.)
A couple of months before the cruise, Little Man and I were engaged in a spirited round of En Garde, Mister! when he rolled onto his back as part of some fancy martial-arts move. I poked him in the belly and was surprised at how big and soft it had become.
What the …?
Little Man had become Tubby Man.
Up until a month or two earlier, he’d been living on canned cat food that’s primarily meat and organ meat. There’s rice in some of the flavors, but not much. For variety, Chareva also fed him sardines, mackerel and tuna.
Then she found a brand of dry cat food that brags No Corn, Wheat or Soy, No Artificial Colors, Flavors or Ingredients on the label. Little Man liked the stuff, so she put it out along with the canned food. Over time, he ate less of the canned food and more of the dry food.
So when I found myself poking a newly-rotund cat belly, I checked the ingredients on the bag of dry food. The first ingredient listed is chicken. That’s good. The next three ingredients are pea powder, barley and brown rice. Well, I wouldn’t call those bad, but it’s clear the dry cat food is considerably more carb-laden than the canned stuff.
I wondered to myself, Did Little Man become Tubby Man because we inadvertently jacked up the carbohydrate content of his diet?
Naaaawww, that can’t be. Legions of internet cowboys have informed me (and everyone on the Fat Head Facebook group) that macronutrients are irrelevant. If you get fat, it’s because you eat too @#$%ing much, too @#$%ing often, period. It’s a simple matter of ingesting too many calories.
Therefore, it was obvious that our Little Man – who for nearly two years had exercised the willpower to limit his calories and maintained a sleek, feline body as a result – was developing a serious flaw in his character. He’d become a glutton without any of us noticing until it was too late. I don’t track his daily activity, but I’ll bet he was also getting lazy and moving less … fewer unexplained mad-dashes around the house and across the top of all the furniture, perhaps.
Anyway, despite being assured by legions of internet cowboys that macronutrients have nothing to do with weight gain, we put the dry food back in the pantry and started feeding him the canned meat again. A month later, he was looking sleek. Had to be a coincidence, of course. I can only guess that somewhere around the time we put the pea-barley-rice dry food away, he happened to recognize himself in a mirror, was disgusted by his tubby appearance, and put himself on a diet.
When we went on the low-carb cruise, we boarded the dogs at a kennel but let Little Man stay at home. Chareva filled a big dispenser with the dry cat food and put out several dishes of water. A friend of Chareva’s also dropped by a couple of times to check on him after feeding our chickens.
Well, wouldn’t you know it … when we returned home eight days later, Little Man was turning into Tubby Man again. I’m not going to chalk it up to a character flaw, since he’d been disciplined enough to eat less and lose weight before we left for the cruise. The obvious explanation this time was emotional eating. The poor cat probably felt abandoned and unloved when we left him home alone, so he comforted himself by eating too much. As Dr. Oz once said about Oprah, “She isn’t really craving food; she’s craving love.” If Little Man had opposable thumbs, he probably would have picked up the TV remote and spent hours watching chick flicks while stuffing himself with the pea-barley-rice food.
But we’ve been back for more than a week, and he’s not engaging in emotional over-eating anymore. He’s even trimmed down noticeably. It has to be because he feels loved and supported again now that we’re home. It can’t have anything to do with the fact that he’s back to a meat-and-fish diet … because as legions of internet cowboys have assured me, macronutrient ratios don’t have anything to do with gaining or losing weight.
102 Comments »
We interrupt our normally scheduled blogging to bring you this commercial announcement.
Mother’s Day is May 10th. We have about 20 of these left:
No, not 20 Charevas … she’s one of a kind. We have about 20 of the Cool Moms Cook With Butter aprons left. They’re available in the Fat Head store.
Or you could just send your mom a nice card.
3 Comments »