Notes
🔍

Minkowski Distance

When a model like k-nearest neighbors says two examples are “close,” it needs a precise way to measure that closeness. Minkowski distance is a flexible distance formula that can behave like several familiar distances depending on one setting.

What it is (and the key knob)

Minkowski distance measures the distance between two feature vectors by raising coordinate-wise differences to a power p, summing them, then taking the p-th root. For two points x and y:

d(x, y) = ( Σ_i |x_i - y_i|^p )^(1/p)

The parameter p controls the geometry:

  • p = 1: Manhattan (L1) distance (grid-like movement).
  • p = 2: Euclidean (L2) distance (straight-line distance).
  • p > 2: increasingly emphasizes large coordinate differences (big gaps dominate).

Why it matters in supervised learning (especially k-NN)

In k-NN classification or regression, the distance metric decides which training points become your “neighbors,” so it directly changes predictions. Larger p makes the model more sensitive to a single feature with a big mismatch; smaller p spreads influence more evenly across features. If features aren’t scaled (e.g., “income” in dollars vs. “age” in years), Minkowski distance will mostly reflect the largest-scale feature, which can quietly break the neighborhood logic.

Practical examples and where you’ll see it

  • Credit scoring: with p=1, many small differences across several attributes can outweigh one large difference, which can be desirable when risk accumulates additively.
  • House price prediction (k-NN regression): with p=2, a large gap in square footage can dominate “closeness,” aligning with how buyers perceive big size differences.

In scikit-learn, you’ll encounter it via KNeighborsClassifier(metric="minkowski", p=2) (Euclidean) or p=1 (Manhattan).

Minkowski distance is a family of Lp norms that measures distance between two feature vectors as \(\left(\sum_i |x_i - y_i|^p\right)^{1/p}\) for \(p \ge 1\). It generalizes Manhattan distance (\(p=1\)) and Euclidean distance (\(p=2\)). It matters because distance-based supervised methods like k-NN depend on this choice to define “nearest” neighbors, directly shaping predictions.

Imagine you’re choosing the “closest” coffee shop. Sometimes you care about straight-line distance. Other times you care about how many blocks you must walk. Minkowski Distance is a flexible way to measure “closeness” that can behave like either of those, depending on a setting you choose.

In supervised learning, especially in k-nearest neighbors, the model predicts something by looking at the most similar past examples. Minkowski Distance is one of the rulers it can use to decide which examples count as “nearest.” Changing the setting changes what “similar” means, which can affect the model’s predictions.