Notes
🔍

Confusion Matrix

A model can have a decent accuracy score and still be making the “wrong kind” of mistakes. A confusion matrix is the simple table that shows you exactly which classes your classifier gets right, and which ones it mixes up.

What it is and how to read it

A confusion matrix compares true labels (what the data says) to predicted labels (what the model says). For binary classification, it’s a 2×2 grid:

  • True Positive (TP): predicted positive, actually positive
  • False Positive (FP): predicted positive, actually negative (a “false alarm”)
  • True Negative (TN): predicted negative, actually negative
  • False Negative (FN): predicted negative, actually positive (a “miss”)

For multi-class problems (like classifying emails into “spam”, “promotions”, “social”), it becomes an N×N table where the diagonal are correct predictions and off-diagonal cells show specific confusions (e.g., “spam” mislabeled as “promotions”).

Why it matters in supervised learning

The matrix is the raw material for many key metrics: precision (TP vs FP), recall (TP vs FN), F1, and class-specific error rates. It also reveals asymmetric costs: in disease diagnosis, FN can be far worse than FP; in fraud detection, too many FP can overwhelm investigators.

How you’ll use it in practice

  • Spot which classes need more data or better features (e.g., churn vs “at-risk” customers getting confused).
  • Choose a decision threshold to trade off FP vs FN when probabilities come from models like LogisticRegression.
  • Diagnose imbalance: a model predicting the majority class can look “accurate” while the matrix exposes near-zero TP.
from sklearn.metrics import confusion_matrix
cm = confusion_matrix(y_true, y_pred)

A confusion matrix is a table that counts how a classifier’s predicted labels compare to the true labels, with entries for true positives, false positives, true negatives, and false negatives (and their multiclass generalization). It matters because it exposes which error types dominate and directly supports key evaluation metrics such as accuracy, precision, recall, F1-score, and specificity.

Think of a confusion matrix like a report card for a sorting job. Imagine a mailroom that must sort letters into “spam” and “not spam.” After a day, you count four things: spam correctly caught, spam that slipped through, good mail wrongly flagged, and good mail correctly delivered.

A confusion matrix does the same for an AI classifier. It’s a simple table that compares what the model predicted versus what was actually true. It helps you see not just how often the model is right, but what kinds of mistakes it makes—like missing fraud versus falsely accusing honest customers.