Notes
🔍

Bayes Theorem

When you learn something new—like a medical test result or a spammy-looking email—you naturally update your belief about what’s going on. Bayes Theorem is the clean mathematical rule for doing that belief-updating in a consistent way.

What Bayes Theorem says

Bayes Theorem connects two conditional probabilities: “probability of a cause given an effect” and “probability of an effect given a cause.” In its common form:

P(A | B) = P(B | A) * P(A) / P(B)

Here’s the meaning in plain terms:

  • P(A) is the prior: your belief about A before seeing B.
  • P(B | A) is the likelihood: how compatible the evidence B is with A.
  • P(B) is the evidence (normalizer): how common B is overall.
  • P(A | B) is the posterior: your updated belief after seeing B.

A relatable example (why base rates matter)

Suppose a disease affects 1% of people. A test is 99% sensitive (positive if sick) and 95% specific (5% false positives). If you test positive, Bayes tells you the chance you’re actually sick is not 99%—it’s closer to about 17%, because false positives among the many healthy people add up. Bayes forces you to combine the test accuracy with the base rate (P(A)).

Why it matters in AI/ML

Many ML models are “Bayesian at heart”: they update beliefs about labels or parameters using data.

  • Naive Bayes classifiers use Bayes to compute P(class | features) for spam detection, sentiment, and document tagging.
  • Bayesian inference (e.g., in probabilistic programming) treats model parameters as uncertain, producing posterior distributions instead of single best guesses.
  • It underpins calibrated uncertainty, crucial in medical AI and risk-sensitive decisions.

Bayes Theorem is a probability rule that updates the probability of a hypothesis after observing evidence: P(H|E) = P(E|H)P(H)/P(E). It formalizes how prior beliefs (P(H)) combine with data likelihood (P(E|H)) to produce a posterior belief (P(H|E)). It underpins Bayesian inference in AI/ML, enabling uncertainty-aware learning. Example: a spam filter updates the probability an email is spam given observed words.

Imagine you’re a detective: you start with a hunch about who did it, then you find a clue, and you update your hunch. Bayes Theorem is the rule that tells you how to update a belief when new evidence shows up.

In statistics and AI, it helps answer questions like: “Given that an email contains certain words, how likely is it to be spam?” You begin with a general guess (how common spam is), then adjust it using the evidence (the words you see). This is why Bayes Theorem is often described as “learning from data” in a very direct, common-sense way.