Highest threshold that reaches a recall target
Medium14 min10 hidden tests
A hospital's readmission model scores every discharged patient, and patients with a score at or above a threshold get a follow-up call. The care team must reach at least target recall: the share of patients who were later readmitted and got a call. A higher threshold means fewer calls, so they want the highest threshold that still meets the target. Write threshold_for_recall(scores, labels, target) that returns that threshold and how many patients it flags.
Set the recall target
highest threshold with recall ≥ targetLowering the threshold from the top, recall goes 0.9 → 0, 0.85 → 0, 0.75 → 0.25, 0.55 → 0.5, 0.4 → 0.75. The highest threshold that reaches 0.75 is 0.4, and it flags 5 patients.
Threshold 0.4, 5 flagged.
Examples
Three of the four readmitted patients must get a call
threshold_for_recall([0.85, 0.1, 0.4, 0.9, 0.55, 0.3, 0.75, 0.2], [0, 1, 1, 0, 1, 0, 1, 0], 0.75)
returns (0.4, 5)
The tie at 0.7 flags both patients, for a recall of 0.75
threshold_for_recall([0.95, 0.7, 0.7, 0.4, 0.6, 0.2], [1, 1, 1, 1, 0, 0], 0.5)
returns (0.7, 3)
Requirements
scoresis a list of floats andlabelsa list of 0 or 1 of the same length (1 means readmitted), with at least one 1.targetis a float above 0 and at most 1.- A patient is flagged when their score is at or above the threshold. Recall is the number of flagged readmitted patients divided by the number of readmitted patients.
- Return a tuple
(threshold, flagged): the highest threshold whose recall is at leasttarget, as a float, and the number of patients of either label it flags, as an int. - Tied scores are all flagged or all left out together.
The idea behind it
A classifier gives scores, and a decision threshold turns them into actions: flag everything at or above it. Moving the threshold trades recall against workload. Recall is the share of real positives that get flagged, so it only depends on the positives' scores. Lowering the threshold to the k-th highest positive score catches at least k positives, more when scores tie. So the highest threshold that reaches a recall target is the k-th highest positive score for the smallest k with k / n_pos >= target. Any higher threshold catches fewer. Negatives do not change the threshold, but they set the cost: every negative at or above it is a wasted call. That count, flagged, is what the team plans staff around. Medical screening, fraud review and content moderation often fix recall first and then try to keep this count small, so the step comes up whenever a model goes to production.