Notes
🔍

Kurtosis

Kurtosis is a way to describe the “shape” of a distribution beyond its average and spread. If variance tells you how wide the data are, kurtosis helps tell you how much of that spread comes from rare, extreme values.

What it measures
Technically, kurtosis is based on the fourth central moment: it looks at the average of (x − mean)4, scaled by the standard deviation to the fourth power. Because deviations are raised to the 4th power, values far from the mean get emphasized a lot. That makes kurtosis a useful indicator of tail heaviness (how often extreme outcomes occur) and, indirectly, how “outlier-prone” a dataset is.

Excess kurtosis and interpretation
Many tools report excess kurtosis, defined as kurtosis minus 3, so that a normal (bell-curve) distribution has excess kurtosis 0.

  • Positive excess kurtosis: heavier tails and more extreme values than normal.
  • Negative excess kurtosis: lighter tails and fewer extremes than normal.

Concrete examples

  • House prices: a city with occasional ultra-luxury sales can show high kurtosis even if most homes cluster tightly.
  • Exam scores: if almost everyone scores near the middle with very few perfect/zero scores, kurtosis may be low (light tails).
  • Medical lab values: rare but severe measurement spikes (true events or sensor glitches) increase kurtosis.

Why it matters in AI/ML
Kurtosis flags when assumptions like “errors are roughly normal” may fail. Heavy tails can make models sensitive to outliers, destabilize gradient-based training, and distort metrics like MSE. It often motivates robust scaling, outlier handling, or using Huber/quantile losses. In practice you’ll see it in exploratory data analysis via libraries like pandas (DataFrame.kurt()) and SciPy (scipy.stats.kurtosis).

Kurtosis measures the heaviness of a distribution’s tails relative to a normal distribution, often computed from the fourth central moment (excess kurtosis subtracts 3 so normal = 0). High kurtosis indicates more extreme outliers; low kurtosis suggests lighter tails. In AI/ML, it helps diagnose non-Gaussian features and outlier-prone data that can affect scaling, robust loss choices, and anomaly detection. Example: monitoring kurtosis of sensor residuals to flag rare spikes.

Imagine two piles of sand: one is a neat, sharp mound, and the other is flatter but with a few grains scattered far away. Kurtosis is a way to describe that “shape” in data—especially how much the data form a sharp peak and how heavy the “tails” are (the tails are the rare, extreme values far from the average).

In statistics and machine learning, kurtosis helps you notice whether a dataset has unusually many outliers. High kurtosis often means more extreme surprises; low kurtosis means fewer extremes and a more even spread.