Notes
🔍

SHAP (SHapley Additive exPlanations)

When a supervised model makes a prediction, it’s natural to ask: “Which inputs pushed it toward that answer?” SHAP is a popular way to break a single prediction into understandable pieces, without pretending the model is simpler than it is.

What SHAP is doing

SHAP (SHapley Additive exPlanations) assigns each feature a contribution value (a “SHAP value”) for a particular prediction. The contributions add up cleanly: starting from a baseline prediction (the model’s average output), you add each feature’s SHAP value to arrive at the model’s final prediction. The core idea comes from Shapley values in game theory: treat features like “players” in a coalition, and measure how much each feature improves prediction quality when it joins different subsets of other features. This averaging across many feature orderings helps handle correlated, interacting inputs more fairly than simple one-at-a-time feature importance.

Why it matters in supervised learning

SHAP gives local explanations (why this one customer was predicted to churn) and can be aggregated into global patterns (which features drive churn overall). It’s used to:

  • Debug models (spot leakage or nonsensical drivers).
  • Communicate decisions in regulated settings (credit scoring, healthcare risk).
  • Compare models beyond accuracy (prefer models with stable, sensible drivers).

Practical examples and tooling

In spam detection, SHAP might show that “contains ‘free’” and “many links” pushed the score toward spam, while “known sender” pulled it back. In house-price prediction, it can quantify how “square footage” and “neighborhood” raised the estimate relative to the baseline. In practice you’ll see the Python shap library (e.g., TreeExplainer for XGBoost/LightGBM) and visualizations like summary plots and force plots.

SHAP (SHapley Additive exPlanations) is a model-agnostic explanation framework that assigns each feature a Shapley value: an additive contribution to a specific prediction, grounded in cooperative game theory and consistent across features. It produces local attributions that can be aggregated into global feature importance. SHAP matters because it makes supervised model decisions auditable and comparable across models, supporting debugging, bias detection, and regulatory justification.

Imagine a group of friends splitting a restaurant bill fairly. They look at who ordered what and decide how much each person “contributed” to the total. SHAP (SHapley Additive exPlanations) does a similar thing for an AI prediction: it splits the final result into “credit” (or “blame”) for each input, like age, income, or past purchases.

This matters because many supervised learning models can feel like black boxes. With SHAP, you can explain why a loan was approved, why an email was flagged as spam, or why a medical risk score was high—using clear, feature-by-feature reasons.