Log loss that never returns infinity
Easy8 min10 hidden tests
A spam filter outputs a probability that each email is spam, and the team scores it with binary log loss. Some probabilities are exactly 0 or 1, and log(0) is minus infinity, so one overconfident email can turn the whole score into infinity or nan. Write log_loss(y_true, p, eps) that clips every probability into [eps, 1 - eps] and returns the mean log loss.
Drag each email’s probability
loss = −mean(y log p + (1 − y) log(1 − p))Email 1 costs −log(0.8) = 0.223; email 2 costs −log(1 − 0.3) = 0.357. Mean log loss 0.2899.
Mean log loss 0.2899.
Examples
Two fairly good predictions
log_loss([1, 0], [0.8, 0.3])
returns 0.289909
Two confident wrong answers cost about 16.1 each after clipping
log_loss([1, 0], [0.0, 1.0])
returns 16.1181
Requirements
y_trueholds n labels, each 0 or 1.pholds n probabilities from 0 to 1, andepsis a small positive float with a default of 1e-7.- First clip each probability into
[eps, 1 - eps]. Then the loss is-mean(y * log(p) + (1 - y) * log(1 - p)), with the natural log. - Return a finite float.
The idea behind it
Binary log loss, also called binary cross-entropy, scores predicted probabilities. For one example with label y and predicted probability p of label 1, the loss is -(y * log(p) + (1 - y) * log(1 - p)). Only one of the two terms is active: -log(p) for a positive example and -log(1 - p) for a negative one. A confident right answer costs almost nothing, and a confident wrong answer costs a lot, without limit: -log(p) goes to infinity as p goes to 0. A model that outputs an exact 0 or 1 then produces infinity, and 0 * log(0) in floating point is nan, which poisons the average. Clipping p into [eps, 1 - eps] caps the cost of one example at -log(eps), about 16.1 for eps = 1e-7. Training code avoids the problem a different way, by computing the loss straight from logits, but evaluation code that gets probabilities needs the clip.