Notes
🔍

Posterior Probability

When a model sees evidence—like the words in an email—it needs a way to update its belief about what’s true. Posterior probability is that updated belief: the probability of a class after you’ve looked at the features.

What it means (and where it comes from)

In supervised classification, the posterior probability is written as P(y \mid x): the probability the label y is correct given the observed input x. It comes from Bayes’ theorem:

P(y | x) = P(x | y) * P(y) / P(x)

Here, P(y) is the prior probability (how common the class is before seeing the example), P(x | y) is the likelihood (how compatible the features are with that class), and P(x) is a normalizing term that makes the probabilities across classes add up to 1.

How Naive Bayes uses it

Naive Bayes computes posteriors by assuming features are conditionally independent given the class, turning the likelihood into a product of per-feature terms. For a spam filter, it estimates:

  • P(spam | words) and P(ham | words), then picks the larger one.
  • Not just a label, but a confidence-like score you can threshold (e.g., flag only if P(spam | x) > 0.95).

Why it matters in practice

Posterior probabilities drive decisions beyond “which class wins”: setting alert thresholds in fraud detection, ranking leads by likelihood to convert, or combining model outputs with business costs. In scikit-learn, Naive Bayes models expose these directly via predict_proba, returning an estimated P(y \mid x) for each class.

Posterior probability is the conditional probability of a hypothesis or class label given observed data, written as P(y|x). In supervised classification (e.g., Naive Bayes), it is computed by combining a prior P(y) with a likelihood P(x|y) via Bayes’ theorem and then normalizing across classes. It matters because predictions, confidence scores, and decision rules (e.g., MAP classification) depend directly on accurate posteriors.

Imagine you’re guessing whether an email is spam. Before you read it, you have a “base hunch” from experience about how often spam shows up. Then you see clues: lots of links, weird wording, a suspicious sender. Your confidence shifts.

Posterior Probability is that updated confidence after you’ve looked at the evidence. In supervised learning, models like Naive Bayes use it to answer questions like: “Given these words in the message, how likely is it to be spam?” It’s a practical way for AI to turn new information into a clearer, more informed prediction.