The accuracy paradox
Imagine a classifier detecting a rare disease that affects 1% of a population. A model that always predicts "no disease," regardless of input, achieves 99% accuracy — and is completely useless, since it never actually detects a single real case. This is the accuracy paradox: on an imbalanced dataset, accuracy can look excellent while the model provides zero real value for the thing it's supposed to detect. This is exactly why precision and recall exist — they measure something accuracy structurally can't.
The four outcomes, and what each actually means
| Predicted Positive | Predicted Negative | |
|---|---|---|
| Actually Positive | True Positive (TP) — correctly caught | False Negative (FN) — missed it |
| Actually Negative | False Positive (FP) — false alarm | True Negative (TN) — correctly cleared |
Precision and recall each focus on a different one of the two mistake types (FP vs. FN) — which is exactly why you can't optimize both simultaneously without a tradeoff, and why neither alone tells the full story.
Precision: of everything I flagged, how much was actually real?
Precision = TP / (TP + FP)
Precision answers: when the model says "positive," how often is it actually right? Low precision means many false alarms — the model over-triggers, flagging things that aren't actually positive. This matters most when a false positive is costly: flagging legitimate email as spam, or falsely accusing a transaction of fraud.
Recall: of everything actually real, how much did I catch?
Recall = TP / (TP + FN)
Recall (also called sensitivity) answers: of all the actual positives out there, how many did the model successfully catch? Low recall means the model misses real cases — exactly the failure mode of the "always predict negative" disease-detection example above, which has 0% recall despite 99% accuracy. This matters most when missing a positive is costly: failing to detect a disease, or missing actual fraud.
The precision/recall tradeoff
Most classifiers output a probability or score, thresholded to produce a final positive/negative decision — lowering that threshold catches more true positives (raising recall) but also lets through more false positives (lowering precision), and vice versa. This is a genuine tradeoff, not a bug to be engineered away — the right balance depends entirely on which mistake (false alarm vs. missed case) is more costly for the specific problem.
F1: a single number balancing both
F1 = 2 × (Precision × Recall) / (Precision + Recall)
F1 is the harmonic mean of precision and recall — specifically chosen over a simple average because the harmonic mean punishes extreme imbalance between the two much more heavily. A model with 100% precision and 1% recall has an average of 50.5%, which sounds reasonable — but its F1 score is barely above 2%, correctly reflecting that the model is nearly useless despite one metric looking perfect.
Common mistakes
- Reporting only accuracy on an imbalanced dataset. As the disease-detection example shows, high accuracy can mask a model that's actually useless for its core task.
- Optimizing only precision or only recall without considering the actual cost of each mistake type. The right tradeoff point depends on the specific problem — there's no universally "correct" balance.
- Using a simple average instead of F1's harmonic mean when precision and recall are very different. A simple average can look deceptively reasonable even when one of the two metrics is near zero.
FAQ
Why can a model have high accuracy but still be useless?
On an imbalanced dataset, always predicting the majority class produces high accuracy while completely failing at detecting the minority class — accuracy doesn't distinguish between "correct because it's genuinely good" and "correct because it never tries."
Is high precision always better than high recall?
Neither is universally better — it depends on which mistake is more costly for your specific problem: false alarms (favor precision) or missed detections (favor recall).
Why does F1 use the harmonic mean instead of a simple average?
The harmonic mean penalizes a large gap between precision and recall far more than a simple average would, so F1 doesn't look artificially good when one of the two metrics is very low.
Compute accuracy, precision, recall, specificity, and F1 from your own confusion matrix with the Confusion Matrix Calculator — entirely in your browser.