Notes
🔍

Variance

Variance is a way to put a number on “how spread out” a set of values is. If two classes have the same average exam score, variance tells you whether most students scored near that average or whether scores were all over the place.

What variance measures
Technically, variance is the average of the squared deviations from the mean. For data points \(x_1,\dots,x_n\) with mean \(\bar{x}\), the population variance is \(\frac{1}{n}\sum (x_i-\bar{x})^2\). In practice, when estimating from a sample, you’ll often see \(\frac{1}{n-1}\sum (x_i-\bar{x})^2\) (the Bessel’s correction) to reduce bias.

Why the squaring?
If you just averaged deviations \((x_i-\bar{x})\), positives and negatives would cancel out. Squaring makes all contributions nonnegative and also makes larger departures count more. The tradeoff is that variance is in “squared units” (e.g., dollars²), which is why people often also report the standard deviation (the square root of variance) for interpretability.

Concrete examples

  • House prices: Two neighborhoods can have the same mean price, but higher variance means more mix of small and luxury homes.
  • Manufacturing: Low variance in bolt lengths signals consistent production; high variance flags quality issues.
  • Medical readings: High variance in blood glucose over time can indicate unstable control even if the average looks fine.

Why it matters in AI/ML
Variance shows up everywhere: feature scaling and normalization depend on it; many models assume noise with constant variance (homoscedasticity); and the bias–variance tradeoff explains why flexible models can overfit (high variance) while overly simple ones underfit (high bias). Algorithms like PCA explicitly seek directions with high variance, and libraries like scikit-learn compute variance for preprocessing and model diagnostics.

Variance measures how spread out values are around their mean, defined as the average squared deviation from the mean (population) or an unbiased estimate using n−1 (sample). It is central in AI/ML because it quantifies uncertainty and noise, underpins standard deviation, and appears in loss functions and probabilistic models (e.g., Gaussian likelihood). Example: comparing feature variances helps decide whether to standardize inputs before training.

Imagine two classrooms that both average 80 on a test. In one class, most students scored close to 80. In the other, some scored 50 and others 100. The average is the same, but the “spread” is very different. That spread is what variance captures.

In statistics, variance is a number that tells you how much values in a dataset tend to differ from the average: small variance means values are tightly clustered; large variance means they’re more scattered. In AI and machine learning, variance helps describe how noisy data is and how much a model’s predictions might change when trained on different samples.