How many articles have you seen on the newspaper and credible websites likes CNN.com that have a million studies referenced? How many authors fill their articles with ‘Research says…’ and link to PubMed? Sure makes the article seems legit, right? But tell me this – How many of those studies have you even read?
Here is the deal. You can write an article about anything under the sun and have studies backing up the claims! Why? Because such is the state of science today. Strongly influenced by politics and money! In this post, I’ll discuss some common mistakes with respect to understanding published literature and explain how you can save yourself from being fooled into believing that all ‘science’ is legit.
Don’t get me wrong. I am not saying that all studies are BS. I’m only saying that you need know how to differentiate between the good and bad studies and more importantly, you need to be able to gauge the relevance of the results to you, your body, your goals and your lifestyle. This is very important because, even for the most health conscious and well-meaning reader, studies and their conclusions can be very misleading if the he/she doesn’t posses a certain level of knowledge/understanding of the subject.
Case 1 – Insufficient understanding
Studies look at multiple (health) markers but researchers very often, in an effort to write attractive and tangible conclusions, decouple the results to a point of oversimplification that it becomes absolutely useless. You probably see many studies that say consumption of whole grains produce drastic improvements in health markers. But what you don’t understand is that when the volunteers were made to eat whole grains, they were also made to dump junk food and sugars, they were (in one way or the other) put in a calorie deficit and they were (in some way) made more active than they previously were! Basically, the control (i.e. the volunteer’s existing diet) is so unbelievably inferior that pretty much any half-decent diet will cause major health improvements. In addition to that, the calorie restriction results in weight loss which also improves health markers. The real deal is in the details and the 3 line conclusions or headlines you read don’t have them!
For example, distance running, you might have heard, is extremely beneficial for heart health. But this is all you hear and read (mostly because this is all you want to hear and read!). If you did take the time to dig deeper into the science behind this statement, it is not really the act of distance running that provides the benefit, but the elevated heart rate due to (any) activity that is responsible for the said benefit. But what is not taken into account is that distance running, in addition to elevating your heart rate, does other things like burning lean muscle and forcing overuse of certain joints. But then you, the running enthusiast, only register the headlines…
‘Research proves running reduces cardiovascular diseases by 40%’
… and jump into your shoes all pumped up for a long run. You invest a lot of time and effort on this because you are under the assumption that you are ‘leading a healthy lifestyle’. But eventually what happens is, due to the negatives associated with chronic distance running, you end up weak and skinny fat with achy joints. But by this time you have developed an emotional bond with running and you view running as the be-all-end-all of fitness that you are practically unable to look at your issues objectively! How many times have you complained of joint/ligament/muscle pain? And how many times have you said/thought… ‘Well, if you’re a runner, you will get injured! Its all part of the game!’. Once again… the real deal is in the details and headlines don’t have them!
Just to prove that I’m not randomly hating on distance running, let’s look at another example. I’m sure you’ve heard that coconut oil is magic and that it increases HDL, decreases LDL, increases metabolism, cures eczema and heals the gut. Let’s say you read the abstract and the conclusion says…
‘Test subjects who consumed 2 tablespoons of coconut oil every morning in an empty stomach lost more weight and had better blood lipids (HDL, LDL and Triglycerides) than the control group.’
Well, you think there is truth to this since you have read the study (only abstract of course) and this is perfect for you right now ‘cos your goal is weight loss and you don’t seem to have time to workout. So you drive to the nearest health store and buy yourself a can of extra virgin cold pressed coconut oil and start consuming 2 tablespoons every morning. A couple of months go by and you realize you’ve actually gained 3-5 lbs. WTF! How did that happen? You took the suggested dose per the advice in the paper! How? Holly hell! How? Well, like 99.99% of people, you didn’t read the paper and so you have no idea what else these test subjects did. You added 2 tablespoons of coconut oil (~ 250 cals) to your existing calorie intake but didn’t really remove anything else. What you didn’t do but the volunteers probably did was to manipulate energy intake (via exercise or food restriction) to ensure they were in a calorie deficit. The paper would have discussed their bodyweights, RMR, BMR, activity, maintenance calorie requirements and actual calorie intakes, but you didn’t read the paper! And once again… the real deal is in the details and conclusions don’t have them!
Case 2 – Neglecting the N=1 factor
Sure studies tell us whats good and whats bad in general and they can be useful in understanding a phenomenon or a concept. But that is really where it stops. The exact outcome of the study has nothing to do with you. A diet that produces 4 lb/week of fat loss on test subjects does not mean you will see the same amount of fat loss every week. There are differences between you and the people enrolled in the study – from lifestyle to body temperature to leptin sensitivity to receptor function to fat mobilization to mitochondrial activity!
And this is precisely why I find it funny when people cite a study and use that as a reason for doing or not doing something. I don’t care if there are 200 studies that say oatmeal is awesome. It absolutely messes with my gut and that is enough reason for me to stay the heck away from it! I don’t care if there are zero studies that prove elimination of grains improves sleep quality. I have solid n=1 evidence that I sleep better when my diet is grain free and that is worth a million studies! And I don’t care what you say ‘cos what I eat now is a result of extensive self-experimentation with respect to food quality, food quantity, meal timing, meal frequency etc. and you shouldn’t care about what I say if you have n=1 evidence that something works or doesn’t work for you! But, as obvious as it sounds, be very objective and true when experimenting with yourself. If a food or exercise doesn’t sit well, it doesn’t matter how much you love it, its got to go!
Case 3 – Assuming all studies are legit
Well, they’re not. They never were and they never will be.
Realize that it isn’t cheap to perform a study and every study that is done needs to be funded. Scraping just the surface – a team of surgeons/physicians/scientists/researchers/professors need to be hired, volunteers needs to be compensated, arrangements for the study (labs, rooms, food, testing facilities etc.) need to be made, animals need to be bred/bought/mobilized, benchmarks need to be set, tests need to be performed at various time intervals, analysis needs to be done, papers need to be written and publications need to be approved! And who do you think will fund all this? If you said the government, you get a $100 fine, you miss a turn and you don’t get to pass through Go! The correct answer is – people who have vested interest in the study. In other words, bigass food, medical and pharma companies!
Companies like Quaker will sponsor studies that prove oatmeal is good for health and companies like Monsanto will fund studies that say soy is a super food and Pharma companies will (abundantly) fund studies that prove statins save lives. So, now, it might not surprise you to know that most studies…
- are initiated with the end result already decided.
- are rigged by tweaking the study design in a way that the results, though unfavorable to the funding company, appear to be favorable.
- have false conclusions that have absolutely nothing in common with the results obtained.
- are short term studies which have no relevance towards long term health or are done on animals and have no carryover to human health.
So very clearly, one cannot make rigid dietary recommendations or choices purely based on scientific evidence. Then how does one know if a particular food/activity is good or bad?
1. Read the entire study in detail before you make changes or don’t make any changes.
2. If you don’t have the patience or time or capability or capacity to read and understand studies, talk to someone who does.
3. If you are convinced that the results of a study might benefit you, make the change only as long as it wont affect you adversely.
4. If you have made the change, listen to your body! And be objective about it. Always remember that studies don’t mean anything unless the change benefits YOU and if it doesn’t benefit you, throw it out.
And since people seem to understand pyramids better…
Listen to your body.
Monitor and track progress.
Make decision to keep or dump change.
For more information on this topic be sure to read this post by J. Stanton on Gnolls.org and watch this video (Science For Smart People) by Tom Naughton.
Thats it for today folks! Have an awesome weekend and please share the post so your friends and family are in the know too!
Stay sane and safe ‘cos its a messed up world out there!