Is a flagged payment really fraud
Easy8 min10 hidden tests
A payments company runs a fraud detector on every transaction. The detection rate is the share of fraud it flags, and the false alarm rate is the share of legitimate transactions it flags by mistake. Fraud is rare, so the share of flags that are real fraud can be much lower than either rate suggests. Write p_fraud_given_flag(base_rate, detection_rate, false_alarm_rate) that uses Bayes' rule to return the probability that a flagged transaction is fraud.
Change the rates
P(fraud | flag) = d · b / (d · b + f · (1 − b))Of 1,000 transactions, about 10 are fraud and 9 of them get flagged; 990 are legitimate and 50 get flagged by mistake. So of 59 flags, 9 are fraud. Exactly: 0.90 × 0.01 / (0.90 × 0.01 + 0.05 × 0.99) = 0.1538.
P of fraud given a flag: 0.1538.
Examples
Only about 15% of flags are fraud, because legitimate payments far outnumber fraud
p_fraud_given_flag(0.01, 0.9, 0.05)
returns 0.153846
With more fraud in the mix, most flags are real
p_fraud_given_flag(0.2, 0.9, 0.05)
returns 0.818182
Requirements
base_rateis P(fraud),detection_rateis P(flag | fraud) andfalse_alarm_rateis P(flag | legitimate). All three are floats from 0 to 1.- Return P(fraud | flag) as a float.
- P(flag) = detection_rate * base_rate + false_alarm_rate * (1 - base_rate).
- When P(flag) is 0, nothing is ever flagged, so return 0.0.
The idea behind it
Bayes' rule turns a probability around. A detector's rates are measured given the truth: P(flag | fraud) and P(flag | legitimate). The review team needs the reverse, P(fraud | flag):
P(fraud | flag) = P(flag | fraud) P(fraud) / P(flag)
The denominator counts every way a flag happens, from fraud and from legitimate payments:
P(flag) = P(flag | fraud) P(fraud) + P(flag | legit) (1 - P(fraud))
When fraud is rare, the second term dominates. A small false alarm rate times a huge pool of legitimate payments can outnumber a high detection rate times a tiny pool of fraud. This is the base rate fallacy, and it is why precision, the share of flags that are right, collapses on rare classes even for a model with strong rates. The same arithmetic applies to medical screening, spam filters, content moderation and any alert system. Teams use it to predict how many false alarms a queue will hold before they deploy a model.