Class Weighting

When one kind of mistake is much worse than another—like missing a fraud case versus flagging a legitimate purchase—you want the model to “care more” about the costly errors. Class weighting is a simple, direct way to bake that preference into training.

What class weighting does

Class weighting assigns a larger penalty to errors on certain classes by multiplying their contribution to the model’s loss function. In a binary classifier, you might give the positive class (e.g., “fraud”) a higher weight so that misclassifying fraud as non-fraud increases the loss more than misclassifying non-fraud as fraud. The optimizer then shifts the learned parameters to reduce those higher-cost mistakes, which typically moves the decision boundary toward the majority class and increases attention to the minority or high-cost class.

Where you see it in practice

• Disease screening: weight “disease” higher to reduce false negatives.
• Credit default: weight “default” higher because missed defaults are expensive.
• Spam detection: weight “not spam” higher if false positives are unacceptable.

In scikit-learn, many models accept class_weight (e.g., LogisticRegression, SVC, tree-based models). A common baseline is class_weight="balanced", which sets weights inversely proportional to class frequency:

from sklearn.linear_model import LogisticRegression
clf = LogisticRegression(class_weight="balanced")

Why it matters (and what it doesn’t do)

Without class weighting, a model trained on skewed data can minimize loss by “playing it safe” and predicting the majority class too often, producing misleadingly high accuracy. Class weighting changes training incentives, but it doesn’t automatically pick the best operating point: you still control the final trade-off via the classification threshold and should evaluate with cost-aware metrics (like precision/recall or expected cost).