Whether we realize it or not, we have all experienced the phenomenon of bad science.
“Coffee Will Kill You and Your Entire Family” a headline in the local newspaper reads. One week later, you see a different headline bragging about how a “New Study Shows Coffee Can Increase Your Lifespan.” Many people who see these constant conflicting messages become, understandably, huffy about the topic of nutrition because things always seem to be changing.
As a dietetic intern and now registered dietitian, I have heard the phrase, “they used to say eggs were bad for us, and now they say they’re healthy” in an accusatory tone about 30,000 times. Learning my title of RD triggers some people, and they often raise their eyebrows, spit their egg anecdote at me, and act like I personally took their sweet, sweet omelets from them decades ago, only to change my mind later.
The reason researchers vilified eggs way back had to do with their high cholesterol content. At the time, all cholesterol seemed unhealthy, especially for those with heart problems. Now we know that dietary cholesterol actually does not affect our heart and blood vessel health as much as we thought. Our liver makes most of our blood cholesterol, and so the bulk of it does not come from dietary cholesterol anyway. The liver actually produces LDL (or “bad”) cholesterol in the presence of saturated and trans fats, which is why the USDA recommends people consume diets lower in saturated fat.1,2
This isn’t a plug to start counting your saturated fat grams; I’m just trying to defend eggs and explain why people were told to reduce their consumption back in the day.
Eggs are super nutrient-rich, and they are a cheap, accessible form of animal protein. They contain lots of vitamins and minerals, and I just think they’re delicious. This is all beside the point, though. What causes foods to be revered one day and then weeks, months, or years down the road, they are seen as evil?
I blame the practice of what author and doctor Ben Goldacre calls “bad science”.
What is Bad Science?
Bad science, in my mind, alludes to the way in which “health professionals”, actual health professionals, journalists, and regular people squander scientific and academic journals to make their point in a way that doesn’t actually make genuine scientific sense given enough critical thinking on the subject.
One of the most common bad science examples occurs when a journalist or anchor takes a single point from a study that seems exciting and alluring, and uses it as a headline to attract attention. Unfortunately, proper context goes out the window, especially when people only skim the headline. Headlines are crafted to be memorable and profitable, and they are a terrible place to get the information you will base your thoughts and beliefs upon.
For example, a headline might say that eating 3 red m&m’s per day is associated with lower rates of insomnia. The headline, and even the article itself, might not tell you that this trial only had 2 individuals eat 3 red m&m’s over a 1 night span. Can we really associate red m&m’s with reduced insomnia based on this tiny study? That doesn’t seem like enough evidence to me.
Obviously, my example exaggerates the idea of crafting a head-turning headline from one small-scale study. However, the exaggeration ends when I say that that many of us get our information, and form our beliefs, from sketchy studies and news articles like the example above. We take what we read online or see on the news at face value, and we don’t bother to try to really understand a situation.
Additionally, news articles often intentionally leave out critical information that can help us understand and interpret the information in a much more comprehensive and relevant way. The details may not feel as exciting, but they aid in genuine comprehension of what we read or hear.
News networks have a major motivation to come up with buzz-worthy headlines that catch our attention. Sponsorships, notoriety, and the almighty dollar lie on the other side of piquing peoples’ curiosity and getting them to come back time and time again. As do websites, social media accounts, and any other businesses vying for your attention.
So again, there are an almost unlimited number of ways researchers, journalists, and internet people could use data to mislead you, allowing you to form your beliefs on a misinterpretation of data or faulty data in itself. We will focus on just three ways that bad science can manifest itself, but do keep in mind that this list could continue infinitely.
Bad science sometimes means taking complicated, complex subjects and reducing them to a single, simple, and catchy point. Many articles conveniently leave out relevant, key factors like confounding variables. Confounding variables are elements that affect a study’s results, but aren’t what researchers are aiming to study.
An example of this might look like “one group of participants ate an extra serving of fruit per day for three months, and they had significantly lower rates of arthritis.” Okay, great! We know fruit provides lots of nutrients, and now we have proof that fruit cures arthritis, right? Not exactly. This excerpt doesn’t tell us how the processes and procedures of the study, or who made up each group.
What if the group that ate an extra serving of fruit per day had mostly younger individuals? And what if the group that didn’t eat extra fruit and suffered more arthritic symptoms, had mostly older individuals? This group would have a higher likelihood to experience arthritis anyway, whether or not they ate extra fruit.
Also, did the researchers take the the sexes of participants into account? The CDC states that women show higher rates of arthritis than men4. So if the group that ate extra fruit contained more women than, maybe the fruit did not actually play a part in reducing arthritic symptoms. Again, perhaps this group was much more likely to develop arthritis whether or not an extra serving of fruit was ingested over the span of the study.
If the study, or news article writing about the study, didn’t go into detail about the demographics of each group, or if they didn’t distribute participants randomly to each group, there are tons of confounding factors that are likely to occur.
Experimental studies, where everything possible in the subjects’ lives is controlled for (everyone fed the same foods, everyone gets the same amount of sleep, etc.) are the best way we have at establishing what variable actually affects the outcome. Even then, there will always be confounding variables, because people, and their bodies, differ in thousands of ways.
Many peer-reviewed studies will discuss potential areas of bias and possible confounding factors, but the vast majority of news stories will not include those while talking about promising new studies that show drinking cherry juice can cure cancer. Confounding variables, and other potentially conflicting information, aren’t exciting enough to take up valuable newspaper or magazine space, but they can still make a huge impact on our understanding of science and life as we know it.
Drug and supplement companies have an interest in conducting studies that yield favorable results for their product. That only makes sense- as the maker of a product that could make your company millions of dollars, studies that show it may not be effective or safe can stand in your way.
As a result, the researchers conducting these studies face pressure to somehow show that a drug, or other product, is safe and effective compared to placebo or the competition.
Funding bias means that studies are much more likely to show a positive outcome for the sponsor’s interests when the company that makes the product pays for the study. Researchers can intentionally or unintentionally skew results in a drug company’s favor in any number of ways, but some of the most common include the following:
- Comparing the drug in question to a competitor at a known ineffective dose
- Contrasting the drugs to a treatment that is known as ineffective
- Comparing the drug in question to a competitor at too high of a dose, so that the group taking the competitor will show more side-effects
- Publishing only the positive results and burying the negative ones
If you read something that seems too good to be true, see if you can get your hands on the original research article. Unfortunately, there are often payment barriers where journals will give you the article in exchange for $30 *eye roll*, but you may find an original PDF of it for free if you look. hard enough. If the study is funded by a drug, supplement, or other special interest company, take the results with a relatively large grain of salt. Maybe more like a brick of salt.
If you aren’t the type to dive right into a research study, that’s okay! I completely understand that isn’t how everyone wants to spend their precious free time. In this case, the best advice I can give you is to carry a healthy amount of skepticism about everything you read and hear and ask probing questions when you can.
The Conscious or Subconscious Belief That Correlation = Causation
I heard the phrase “correlation does not equal causation” about a hundred times throughout the course of my Psychology 101 class in my freshman year of college. Those of us in the health or science fields have been told this phrase time and time again, and I think we have heard it so many times that we numbly nod in agreement without always stopping to really think about it.
When a study establishes correlation between two factors, like eating lots of fruits and vegetables is correlated with lower rates of heart disease, stroke, and cancer, it’s saying that there is an association. Yep, people who regularly enjoy fruits and vegetables do tend to be healthier individuals, but we can’t say that fruits and vegetables cause health in the same way we can’t say not eating them causes disease.
This goes back to the confounding variable section earlier. Maybe people who eat lots of fruits and vegetables also get lots of exercise, have steady incomes, less stress, and have a stronger social network. Maybe their lower rates of these diseases have to do with all of these factors combined instead of simply just the fruits and vegetables.
This is why we can’t say that not eating fruits and vegetables cause cancer. There are very few examples in the world of science and health where we can say with full certainty that x causes y. One notable exception is that smoking does cause cancer.3,4 The evidence for this had to be so thorough and watertight, that it was undeniable.
Even if a research paper never says that the food or herb in question causes something else, that doesn’t mean journalists and people in the public don’t interpret it that way. In most cases, the only way to really establish causation is to do an experimental study like we talked about above, where you control for as many variables as possible.
Those studies are costly, difficult to conduct, and are sometimes highly unethical, so they are pretty hard to come by. An alternative option is to have tons of observational data like in the case of smoking, but again, it has to be watertight.
How Does Bad Science Relate to Diet Culture?
The diet industry knows and exploits the fact that most of us do not really try to understand what we read and hear. It loves that many of us form our beliefs based on catchy, memorable headlines, like “the grapefruit diet will help you lose 14 pounds in a week”. You don’t need a degree in Biology, or even a love or passion for science, to dig a little deeper and push back when you see a juicy headline that seems suspiciously enticing.
Bad science also includes supplement salesmen or diet plan developers that cite research studies that actually don’t exist. In Bad Science, Ben Goldacre discusses the time he saw a doctor on TV cite an Australian study that supposedly touted the anti-aging benefits of pomegranate juice. He proclaimed that it can even reduce wrinkles!
Unfortunately, this study was absolutely nowhere to be found.5 It sounds good to cite a study on TV, and if we are being honest with ourselves, very few of us would dig deeper to try to read that study for ourselves. Instead, we might start drinking gallons of pomegranate juice in the name of bouncy skin and a longer lifespan, despite the expense and lack of actual evidence that comes with it.
You can make almost anything sound good and reasonable if you cite an unspecific study most people won’t bother to look into. Or you can also cite legitimate studies, but spin the results to make your product look like an appealing, evidence-based product.
This is what I did to myself when I researched the effectiveness of supplements I was already planning on buying. I looked for “evidence” that confirmed what I already believed, and I selectively ignored the stuff that didn’t align with what I wanted. Diet culture does its best to sound scientific and evidence-based, but most of its claims rely on bad science.
How Can I Avoid Bad Science in Everyday Life?
If you’re looking for a quick answer here, the best advice I can give is to take literally everything you read, including this blog and my posts, with a grain of salt. That’s not to say that I am ever attempting to deliberately misinform or manipulate you; instead, I mean that I have my own biases that have been shaped by my experiences, as do you, and researchers, doctors, and every single person on earth.
Be open-minded, curious, skeptical, and critical of everything you read, see, hear, and think. This doesn’t mean you have to go around disputing everything people say or drive yourself into extreme paranoia. Instead, it means that you don’t jump to conclusions because it tugs at your emotions or “feels” right. It means that you dig into research yourself and learn as much as you can about a given topic before you come to a conclusion.
Have you ever had the experience where you have this harsh feeling or judgement of a person you don’t know very well based one thing another person has told us, only to find out they’re a really lovely individual once you get to know them better? I’d wager we have all had this happen before, and it’s a real-life example of how we can’t get the whole picture of something based on one thing someone else has told us.
Too often we rely on other people to tell us what to think instead of learning how to think for ourselves. We obviously can’t observe all experiments for ourselves and learn everything about everything, so we have to rely on journalists and experts to do that work for us sometimes. But, this does not mean that we should compose our beliefs and values based on one news source or one headline. Including political ones.
If you are into reading research studies and learning things for yourself, keep your eyes out for papers called systematic reviews. These are papers where the researchers scoured every study they could find regarding a certain topic, threw out ones that didn’t meet a rigid set of criteria, and compiled the results. Systematic reviews give us a much more comprehensive view of an issue, instead of relying on a handful of studies that may have been conducted by drug companies or other special interest groups.
Cochrane Reviews are highly-respected, internationally-recognized reviews that are performed with rigorous standards. You can search the Cochrane Library for various health-related topics. They even have Plain Language Summaries, where they translate their clunky, scientifically-worded reviews into resources for the general public. Unfortunately, there aren’t too many reviews for nutrition-related topics yet, but I still believe that knowing about the most reliable resources is hugely important.
If you are interested in learning more about how to read and understand scientific literature, Cochrane also offers Evidence Essentials, a free online course that will help you learn how to interpret studies and papers for yourself. I understand this might not be the most fun use of your time, but I truly believe that understanding how to interpret evidence is a highly valuable skill in work and life.
As Always, A Book Recommendation
The inspiration for this post came from the incredible book Bad Science by Ben Goldacre. Ben Goldacre is a doctor, as well as an outspoken figure on the topic of bad science in regards to nutrition, pharmaceuticals, and other health-related areas of interest.
I listened to this audiobook in just over a day because I was so intrigued by the subject matter. I especially loved when he took an in-depth look at some famous nutritionists (those who get certified by one online course, not registered dietitians) who have made tons of money off of nutrition claims that have no scientific legs to stand on.
Ben Goldacre uses humor and tells stories throughout this wonderful read in a way that makes this book accessible and fun even for people who don’t particularly like science or find research studies very interesting. I highly recommend this book because he provides you with the mental tools you need to think for yourself instead of simply believing things other people (including news reporters) tell you because it aligns with your current beliefs.
As a proud Indie Bound affiliate, I get a small commission on books and products sold using the links on my website. It doesn’t cost you anything extra! I only recommend books that I truly love and believe in, and your purchases on Indie Bound through my website support both my work and small bookstores. Thanks!
- Dietary Fats: Know Which Types to Choose – Mayo Clinic
- The Skinny on Fats – American Heart Association
- Arthritis Risk Factors – CDC
- Smoking and Cancer – CDC
- Cranky to Fashionable in Five Iffy Claims – The Guardian
Thanks for visiting The Diabolical Dietitian! If you’d like to stay up-to-date on the latest posts, please enter your email below to subscribe!