Why Some Fitness Studies Are Wrong
With Great Power Comes Great Responsibility: Common Errors in Meta-Analyses and Meta-Regressions in Strength & Conditioning Research
Not medical advice. For informational purposes only. Always consult a healthcare professional. Terms
Surprising Findings
59% of all effect sizes greater than 3.0 — considered biologically implausible — were caused by the standard error/standard deviation mix-up.
People assume huge gains like ‘11x improvement’ are breakthroughs — but this study shows they’re almost always statistical errors, not science.
Practical Takeaways
When you see a fitness study claiming ‘huge gains,’ check if it’s a meta-analysis — then ask: Did they use standard deviation (not error)? Did they account for repeated measures? Did they compare to a control group?
Not medical advice. For informational purposes only. Always consult a healthcare professional. Terms
Surprising Findings
59% of all effect sizes greater than 3.0 — considered biologically implausible — were caused by the standard error/standard deviation mix-up.
People assume huge gains like ‘11x improvement’ are breakthroughs — but this study shows they’re almost always statistical errors, not science.
Practical Takeaways
When you see a fitness study claiming ‘huge gains,’ check if it’s a meta-analysis — then ask: Did they use standard deviation (not error)? Did they account for repeated measures? Did they compare to a control group?
Publication
Journal
Sports Medicine (Auckland, N.z.)
Year
2022
Authors
D. Kadlec, Kristin L. Sainani, Sophia Nimphius
Related Content
Claims (6)
A big review found that in most top-rated studies about strength and fitness training, about 85% made at least one common math mistake—like using the wrong numbers to show results or including results that seem way too extreme.
In nearly half of the most popular science reviews on strength and fitness, researchers made a math mistake—they used the wrong number to calculate how strong an exercise or supplement’s effect was. This made the effects look way bigger than they really were, and most of the super huge effects (over 3.0) were just due to this error.
A lot of big studies that combine other studies on strength and fitness didn’t properly handle cases where the same people were measured multiple times or when one study had several groups — this made their results look more precise and important than they really were.
A lot of the most popular fitness studies only looked at how people improved on their own after a workout or supplement, not how they did compared to people who didn’t do anything — which might make those treatments seem way more effective than they really are.
When scientists combine results from lots of fitness studies, sometimes they get really big numbers—bigger than 3.0—and this claim says almost 6 out of 10 of those huge numbers are probably mistakes because they used the wrong math formula.