Log Loss

If a classifier tells you “this email is spam,” you usually want more than a yes/no—you want a probability you can trust. Log loss is the scoring rule that rewards models for putting high probability on the correct class and punishes them sharply when they sound confident but are wrong.

What log loss measures

In binary classification with labels y in {0,1} and predicted probability p = P(y=1), log loss (also called cross-entropy loss) for one example is:

loss = -[ y * log(p) + (1 - y) * log(1 - p) ]

If the true label is 1, the loss is -log(p): predicting 0.9 is good (small loss), predicting 0.01 is disastrous (huge loss). The logarithm is what creates that “don’t be confidently wrong” behavior.

Why it matters in supervised learning

Log loss directly trains probabilistic classifiers to produce well-behaved probabilities, not just correct labels. It’s the standard objective behind models like logistic regression and is widely used as an evaluation metric when decision thresholds vary.

Spam detection: a message scored 0.55 vs 0.99 should be treated differently; log loss distinguishes them.
Credit risk: calibrated default probabilities matter for pricing and policy, not just “approve/deny.”
Fraud detection: extreme false confidence is costly; log loss penalizes it heavily.

How you’ll see it in practice

In scikit-learn, you’ll encounter it as sklearn.metrics.log_loss, and many libraries (e.g., XGBoost) optimize related objectives like binary:logistic. A key detail: because log(0) is undefined, implementations clip probabilities away from 0 and 1 to keep training stable.

Log Loss (also called cross-entropy loss or negative log-likelihood) measures how well a classifier’s predicted class probabilities match the true labels by penalizing low probability assigned to the correct class, with severe penalties for confident wrong predictions. It matters because it is the standard objective for training probabilistic classifiers and directly drives both discrimination and probability calibration (e.g., logistic regression, softmax networks).

Think of a weather app that says there’s a 90% chance of rain. If it stays sunny, that’s a much bigger “oops” than if it had said 55%. Log Loss is a way to score a classifier (a model that picks between categories) based on the probabilities it gives, not just whether it was right or wrong.

It rewards confident predictions when they’re correct, but it strongly punishes being confident and wrong. That matters for things like spam filters, medical tests, or fraud alerts, where you want the model’s “confidence” to be trustworthy, not just its final yes/no answer.