Adjusted R-Squared

When you add more features to a regression model, it can feel like you’re “improving” it—because the fit to the training data almost always looks better. Adjusted R-Squared exists to keep that improvement honest by rewarding real explanatory power and penalizing unnecessary complexity.

What it measures (and why it exists)

Regular R-squared measures the fraction of variance in the target that your model explains compared to a simple baseline (predicting the mean). The catch: R-squared never decreases when you add predictors, even if the new feature is pure noise. Adjusted R-squared corrects this by subtracting a penalty that grows with the number of predictors. It’s computed from R-squared, the number of observations n, and the number of predictors p (excluding the intercept): it only increases when a new feature improves fit enough to justify its cost.

How to interpret it in practice

• Use it to compare regression models trained on the same dataset predicting the same target.
• If adding a feature makes adjusted R-squared go down, that feature likely isn’t helping (or is helping too little).
• It’s most natural for linear regression (e.g., OLS); for heavily regularized or non-linear models, it’s less central.

Real-world examples and common pitfalls

In house price prediction, adding “number of nearby coffee shops” might raise R-squared slightly just by chance; adjusted R-squared can drop, signaling it’s not worth keeping. In credit risk regression (predicting loss amount), it helps you avoid bloated models that look good in-sample but generalize poorly. In Python, you’ll see it in statsmodels OLS results as rsquared_adj; scikit-learn reports R-squared but not the adjusted version by default.