Probability Mass Function (PMF)
A lot of data problems in AI/ML start with a simple question: “What outcomes are possible, and how likely is each one?” When the outcomes are countable (like 0, 1, 2, …), the tool that answers that question is the Probability Mass Function (PMF).
What a PMF is
A PMF describes the probability distribution of a discrete random variable. It assigns a probability to each possible value the variable can take. If X is discrete, its PMF is written as p(x) = P(X = x).
- Each probability is between 0 and 1: 0 ≤ p(x) ≤ 1
- All probabilities add up to 1: ∑ p(x) = 1 over all possible x
- Probabilities are for exact values (not ranges): P(X=3), not “between 2 and 4”
Intuition and a quick analogy
Think of a PMF like a “probability menu” for a count-based outcome: it lists every dish (outcome) and its price (probability). Some outcomes get more “mass” than others, but the total mass is always 1.
Practical examples
- Customer purchases: X = number of items bought today; the PMF gives P(X=0), P(X=1), P(X=2), …
- Medical events: X = number of ER visits in a month; often modeled with a Poisson PMF
- Classification outputs: a model’s predicted class probabilities act like a PMF over labels (they sum to 1)
Why it matters in AI/ML
PMFs are the backbone of many discrete models: Naive Bayes, Hidden Markov Models, and language modeling over tokens. Training often maximizes likelihood, which uses PMF values directly; if your probabilities don’t sum to 1 (or assign impossible outcomes nonzero mass), inference and learning break.
Probability Mass Function (PMF) is a function for a discrete random variable that assigns a probability to each possible outcome, with all probabilities nonnegative and summing to 1. It fully specifies a discrete probability distribution and enables computing expectations, variances, and likelihoods. In AI/ML, PMFs underpin models for categorical or count data. Example: a multinomial classifier outputs a PMF over classes for a given input.
Imagine a jar of colored candies: you don’t know which color you’ll grab, but you can list the chance of each color—30% red, 50% blue, 20% green. A Probability Mass Function (PMF) is like that list of chances, but for situations where the outcomes are separate, countable options (like rolling a die: 1, 2, 3, 4, 5, 6).
In statistics and AI, a PMF tells you how likely each possible value is for a “discrete” random event. All the probabilities add up to 1, meaning they cover every possible outcome.