Standardization

When your dataset has features measured in wildly different units—income in dollars, age in years, and clicks per day—models can struggle to “compare” them fairly. Standardization is a simple way to put numeric features onto a common scale without throwing away their relative differences.

What standardization does

Standardization transforms a feature so it has a mean of 0 and a standard deviation of 1 (on the training set). For each value x, you compute z-score: (x − mean) / std. After this, a value of +2 means “two standard deviations above average,” regardless of whether the original feature was dollars or years. This is different from min–max scaling: standardization doesn’t force values into [0, 1]; it centers and rescales based on spread.

Why it matters for supervised models

Many algorithms are sensitive to feature scale because they use distances, dot products, or gradient-based optimization. Without standardization:

• LogisticRegression, linear regression with L1/L2 regularization, and SVM can overweight large-scale features, distorting coefficients and decision boundaries.
• KNN and other distance-based methods can become dominated by one feature (e.g., “income” drowning out “age”).
• Gradient descent can converge slowly or unstably when features have very different magnitudes.

How it’s used in practice

In credit scoring or churn prediction, you standardize numeric columns (balance, tenure, transactions) while handling categoricals separately. Crucially, you fit the scaler on the training data and then transform validation/test data with the same parameters to avoid leakage:

from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import make_pipeline
from sklearn.linear_model import LogisticRegression

model = make_pipeline(StandardScaler(), LogisticRegression())