The Claim
Feature selection conducted on the entire dataset prior to cross-validation introduces greater bias in performance estimation compared to hyperparameter tuning, because it exposes the model to information from the test data, enabling it to capitalize on spurious noise patterns that appear predictive by chance, particularly in high-dimensional datasets with limited sample sizes.
What the research says
Not yet evaluated
We are still looking at what the research says.
These are independent scores, not a percentage. Higher-grade studies count more, so a single strong opposing study can outweigh several weaker ones.
If you pick the most important features using all your data — including the test set — your model might look better than it really is, because it's secretly cheating by seeing data it shouldn't see yet.
See the scientific wording
Feature selection performed on the full dataset before cross-validation contributes more to performance estimate bias than hyperparameter tuning, as selecting features based on the entire dataset—including test data—allows the model to exploit random noise patterns that appear discriminative by chance, especially in high-dimensional, small-sample settings.
What the research says
1 studyStudy: Machine learning algorithm validation with a limited sample size
The study shows that picking features using all the data (including test data) tricks the model into thinking it's better than it really is, and this causes more misleading results than adjusting the model's settings.
Score breakdown, mechanism chain, raw evidence, ideal studies needed & 1 supporting studies
Not medical advice. For informational purposes only. Always consult a qualified healthcare professional before making health decisions.