Notes
🔍

Log-Odds

Probabilities are easy to read, but they’re awkward to model directly with a straight line: they must stay between 0 and 1. Log-odds is the trick that turns a probability into a number that can range over all real values, so a linear model can work cleanly.

What log-odds means

Start with a probability p of the positive class (say, “email is spam”). The odds are p/(1-p): how much more likely “spam” is than “not spam.” Then take the natural log to get log-odds (also called the logit):

logit(p) = log(p / (1 - p))

This mapping is reversible: if you know the log-odds value z, you can recover the probability with the sigmoid:

p = 1 / (1 + exp(-z))

How logistic regression uses it

In logistic regression, the model assumes the log-odds are a linear function of features:

logit(p) = w0 + w1 x1 + ... + wk xk

That’s why coefficients are so interpretable: increasing feature xj by 1 changes the log-odds by wj, which multiplies the odds by exp(wj). For example, in credit scoring, a coefficient of 0.7 on “has prior late payment” means the odds of default are multiplied by exp(0.7) ≈ 2.0, holding other features fixed.

Why it matters in practice

  • It keeps predictions valid: the linear part can be any real number, while the sigmoid converts it into a proper probability.
  • It makes effects additive and stable to estimate with maximum likelihood (what libraries like scikit-learn’s LogisticRegression optimize).
  • It supports clear communication: “odds double” is often easier to explain than “probability increases by 0.08,” especially when baseline risk differs.

Log-odds is the logarithm of the odds of an event, defined as logit(p) = log(p/(1−p)) for probability p. It maps probabilities in (0,1) to real numbers, enabling a linear model to represent class likelihoods. In logistic regression, the predicted log-odds is a linear function of features, so coefficients quantify additive changes in log-odds per unit feature change.

Think of betting on a game. If you say “it’s 3 to 1 that Team A wins,” you’re using odds: how much more likely one outcome is than the other. Log-odds just means taking those odds and putting them on a scale that’s easier to work with: equal chances becomes 0, “more likely” becomes a positive number, and “less likely” becomes a negative number.

In AI, especially logistic regression, log-odds are a handy way to turn clues (like words in an email) into a single score that reflects how strongly the model leans toward “spam” versus “not spam.”