Notes
🔍

Hamming Distance

When your data is made of yes/no choices—like “clicked” vs “didn’t click,” or a set of on/off flags—measuring “how far apart” two examples are can be as simple as counting how many answers disagree. That’s the intuition behind Hamming Distance.

What it measures

Hamming Distance is the number of positions where two equal-length vectors differ. It was originally defined for binary strings, but it applies to any fixed-length sequence where “match vs mismatch” is meaningful. For example, between 101110 and 100010, the distance is 2 because two bit positions differ. Unlike Euclidean distance, it ignores magnitude and focuses purely on disagreement per feature.

How it’s used in supervised learning

In k-Nearest Neighbors (k-NN), the distance metric decides what “nearest” means. If your features are binary indicators (e.g., “has coupon,” “is mobile user,” “visited pricing page”), Hamming Distance makes k-NN compare customers by counting how many indicators differ, which matches the data’s structure. Common supervised scenarios include:

  • Spam detection: presence/absence of certain keywords or header patterns.
  • Customer churn: a vector of behavioral flags (used feature X, contacted support, missed payment).
  • Fraud detection: rule-based signals encoded as 0/1.

Why it matters (and what can go wrong)

If you use a metric like Euclidean distance on binary/categorical encodings, you can create misleading neighborhoods (especially with one-hot features), causing k-NN to vote based on “closeness” that doesn’t reflect real similarity. In scikit-learn, you’ll see this via k-NN’s metric option (e.g., metric="hamming"), which makes the model’s notion of similarity align with mismatch-counting.

Hamming Distance is a distance metric for equal-length discrete vectors (typically binary or categorical) defined as the number of positions at which the corresponding entries differ. For example, between 10110 and 10011 it equals 2. It matters in supervised learning because it provides a well-defined notion of similarity for k-NN and related methods on binary/one-hot features, directly determining nearest neighbors and thus predictions.

Think of two words typed on a phone: “cats” and “cuts.” They’re the same length, and only one letter is different. Hamming Distance is simply the count of positions where two same-length strings (or lists) don’t match.

In AI, this is handy when your data is made of yes/no or on/off choices, like “has a keyword,” “is weekend,” or “clicked ad.” If two customers’ profiles differ in 3 out of 20 such switches, their Hamming Distance is 3. In methods like k-nearest neighbors, that helps decide which past examples are most “similar” for making a prediction.