Precision-Recall Curve
When a classifier outputs a score (like “probability of fraud”), you still have to choose a cutoff to turn that score into a yes/no decision. A precision-recall curve shows what you gain and what you sacrifice as you slide that cutoff up and down.
What it plots (and what it means)
A Precision-Recall (PR) curve plots precision on the y-axis against recall on the x-axis for every possible decision threshold.
- Precision = among the items you predicted positive, how many were truly positive? (low precision means lots of false alarms)
- Recall = among the truly positive items, how many did you catch? (low recall means you missed many positives)
As you lower the threshold, the model predicts “positive” more easily: recall rises, but precision usually falls because more false positives slip in. The curve visualizes that trade-off.
Why it matters (especially with rare positives)
PR curves are especially useful when the positive class is rare (fraud detection, disease screening, churn). In these settings, a model can look great on metrics that are insensitive to class imbalance, yet still generate too many false positives. The PR curve keeps attention on the two quantities you actually care about when positives are scarce: catching positives (recall) without drowning in false alerts (precision).
How it’s used in practice
- Pick an operating point: e.g., “at least 80% recall while keeping precision above 30%” for fraud review capacity.
- Compare models via Average Precision (AP), a common summary of the PR curve (higher is better).
- In scikit-learn, you’ll see
precision_recall_curveandaverage_precision_scoreused with model scores (not hard labels).
A Precision-Recall Curve plots precision (positive predictive value) against recall (true positive rate) as a classifier’s decision threshold varies, showing the trade-off between finding more positives and keeping predicted positives accurate. It matters because it is a primary diagnostic for imbalanced classification, where ROC curves can look overly optimistic; summary metrics like Average Precision depend on it.
Imagine a spam filter that can be set to be “strict” or “lenient.” If it’s too strict, it catches lots of spam but might wrongly block real emails. If it’s too lenient, it lets spam through. A Precision-Recall Curve is a simple chart that shows this trade-off as you slide that strictness setting.
Precision means: when the model says “this is spam,” how often is it right? Recall means: out of all the spam that exists, how much did it catch? The curve helps you pick a setting that fits your goal, especially when the “important” cases are rare, like fraud or disease.