Binary Cross-Entropy Loss

When you train a model to answer a yes/no question—“is this email spam?” “will this customer churn?”—you want it to be confidently right and painfully punished when it’s confidently wrong. Binary cross-entropy loss is the standard way to do that when your model outputs a probability.

What it measures

Binary cross-entropy compares the true label y (0 or 1) to the model’s predicted probability p of class 1. The loss for one example is:

loss = -[ y * log(p) + (1 - y) * log(1 - p) ]

If the true label is 1, the second term disappears and the loss is -log(p): predicting 0.9 is rewarded (small loss), predicting 0.01 is punished (huge loss). If the true label is 0, it becomes -log(1-p). This “log penalty” is why the loss pushes probabilities toward the correct side, not just the correct class.

Why it matters in training

This loss is the negative log-likelihood of a Bernoulli model, which makes it a natural fit for logistic regression and neural nets with a sigmoid output. It produces clean gradients for optimization and encourages probability calibration: a model that outputs 0.8 should be right about 80% of the time.

Where you’ll see it in practice

Spam detection: penalizes “definitely not spam” predictions on actual spam heavily.
Credit default: supports risk-based decisions because outputs are probabilities.
Churn prediction: enables threshold tuning (e.g., contact customers when p > 0.6).

In libraries, it appears as scikit-learn’s log_loss and in deep learning as “binary crossentropy” (often implemented in a numerically stable “with logits” form).

Binary Cross-Entropy Loss (log loss) measures the negative log-likelihood of true binary labels under a model’s predicted probability for the positive class, penalizing confident wrong predictions heavily. For a label y∈{0,1} and predicted probability p, it is −[y·log(p)+(1−y)·log(1−p)]. It matters because it is the standard objective for training probabilistic binary classifiers (e.g., logistic regression, sigmoid outputs), enabling stable optimization and calibrated probabilities.

Imagine a weather app that says there’s a 90% chance of rain, but it stays sunny. That feels like a bigger “mistake” than saying 55% and being wrong. Binary Cross-Entropy Loss is a way to score mistakes like that when there are only two possible answers: yes/no, spam/not spam, sick/healthy.

In supervised learning, a model often outputs a probability for the “yes” class. Binary Cross-Entropy Loss gives a small penalty when the model is confident and correct, and a very large penalty when it’s confident and wrong. This pushes the model to make probabilities that match reality, not just guesses.