Critically evaluating supplements, etc.: systematic reviews, meta-analyses, biases, evidence, etc.

Don’t be fooled by biochemically-plausible-sounding claims!

Critically evaluating supplements, etc.: systematic reviews, meta-analyses, biases, evidence, etc. (https://www.youtube.com/watch?v=NBZmlNxZeMs)

Use biochemistry knowledge with caution!

Things are usually way more complicated than they seem!
- Because pretty much everything’s interconnected in your body, you can almost always draw connections between one thing and another
  - This is awesome, but also makes it easy to develop basically limitless plausible-sounding theories
  - Even if you understand things on paper, things get massively more complicated in a person!

Seek out systematic reviews & meta-analyses

These filter, collate, and try to determine “consensuses” from numerous studies surrounding a topic

But remember: garbage in, garbage out!

The conclusions are only as strong as the studies that go into them
The results can be biased based on which studies are and are not included, and how rigorous those studies were

Seek out trusted sources – and multiple sources

One of my favorite podcasts is Skeptic’s Guide to the Universe. One of the co-hosts, Steve Novella, co-runs a website called Science-based medicine which is a great place to turn if you want to quickly evaluate the evidence (or lack thereof) behind a new (or old) fad, etc. https://sciencebasedmedicine.org

Comparing different studies is inherently difficult

Correlation does not equal causation

Just because #s for both ice cream sales and pool drownings rise and fall together doesn’t mean that ice cream causes drowning!

Look for the hidden variables

Good studies should control for things like diet, exercise, socioeconomic status, underlying or co-existing health conditions, etc.

Judge the quality of the evidence

Be aware of p-fishing/p-hacking/data dredging

The more correlations you look for, the more likely you are to find one just by chance

Don’t confuse statistical significance with clinical/meaningful significance

Is it really worth taking a supplement if it is shown to statistically-significantly increase your sleep 0.5 s/night?

Remember risk (and cost) vs. benefit

Don’t undervalue the power of biases (conscious and unconscious)

Always keep in mind regression to the mean

Things often get better – with or without intervention
People often seek out intervention when things are bad
- When things get better they credit the intervention, even though they would have gotten better anyway

Know the placebo effect is powerful!

This isn’t saying “it’s all in your head” – just that your head affects your body

Some search terms that might help you find things

“Topic” + __________:
- “systematic review”
- “meta-analysis”
- “evidence-based”
- “controversial”
- “consensus”
- “conflicting”
- differing”
- “evidence”
Can also search within article for these terms to see whom they’re citing and go look there