Notes
🔍

Posterior Probability

Posterior probability is what you believe after you’ve seen some evidence. It’s the probability of a hypothesis being true once you’ve updated your earlier belief with new data—like revising your estimate that a house is worth $500k after you see recent nearby sales.

What it means (in Bayesian terms)
In Bayesian statistics, you start with a prior probability (your belief before data), observe data through a likelihood (how compatible the data is with different hypotheses), and end with the posterior probability (your updated belief). Bayes’ theorem ties these together:

posterior ∝ likelihood × prior

More explicitly: P(H | D) (posterior) is proportional to P(D | H) (likelihood) times P(H) (prior), with a normalizing constant so probabilities add up to 1. The key idea is that the posterior balances what you believed before with what the data is telling you.

A concrete example
Suppose 1% of people have a disease (prior). A test is 99% sensitive and 95% specific (likelihood behavior). If someone tests positive, the posterior probability P(disease | positive) is not 99%—it’s much lower because false positives matter when the disease is rare. The posterior is the “real” probability you care about after combining base rates and test accuracy.

Why it matters in AI/ML
Many ML systems are about estimating uncertainty, not just making a single guess. Posterior probabilities enable:

  • Probabilistic classification (e.g., Naive Bayes outputs class posteriors).
  • Bayesian regression, giving a distribution over parameters and predictions.
  • Bayesian neural networks and approximate inference (e.g., variational inference), which quantify uncertainty under limited data or dataset shift.

Without posterior thinking, models often produce confident-looking predictions without a principled way to express “how sure” they are.

Posterior Probability is the updated probability of a hypothesis or parameter value after observing data, computed by combining a prior belief with the likelihood via Bayes’ theorem. It is central in Bayesian inference because it quantifies uncertainty in model parameters and predictions given evidence, enabling principled learning and decision-making. Example: in Bayesian logistic regression, the posterior over coefficients reflects which features are likely influential after seeing labeled training data.

Imagine you’re guessing whether it will rain today. You start with a gut feeling based on the season and the forecast you heard yesterday. Then you look outside and see dark clouds. You update your guess using this new evidence.

That updated guess is posterior probability: the probability of something being true after you’ve taken new data into account. In Bayesian statistics (often used in AI), you begin with a prior belief, then combine it with the evidence from data, and the result is the posterior probability. It’s how models “revise their beliefs” as they learn from observations.