expected_savings_score#

empulse.metrics.expected_savings_score#

Expected savings of a classifier compared to a baseline.

A generic Metric built from the Savings strategy on a plain cost matrix, accepting class- or instance-dependent tp_cost, tn_cost, fp_cost, and fn_cost parameters, plus a baseline argument.

baseline accepts:

  • 'zero_one' (default): a naive model that predicts all zeros or all ones, whichever is better.

  • 'one': a model that predicts all ones.

  • 'zero': a model that predicts all zeros.

  • 'prior': a model that predicts the prior probability of the majority or minority class, whichever is better.

  • array-like: target probabilities of a baseline model.

With 1 being the perfect model (assuming 0 cost is the lowest you can go, for negative costs the savings metric can go higher than 1), 0 being as good as the baseline model, and values smaller than 0 being worse than the baseline model.

See also

savings_score : Cost savings of a classifier compared to a baseline, using hard (thresholded) labels.

expected_cost_loss : Expected cost of a classifier.

Methods

__call__(y_true, y_proba, *, tp_cost=0.0, tn_cost=0.0, fp_cost=0.0, fn_cost=0.0, baseline='zero_one')

Compute the expected savings of a classifier compared to a baseline.

optimal_threshold(y_true, y_proba, *, tp_cost=0.0, tn_cost=0.0, fp_cost=0.0, fn_cost=0.0, baseline='zero_one')

Compute the classification threshold(s) that minimize(s) the expected cost (equivalently, that maximize(s) the expected savings).

optimal_rate(y_true, y_proba, *, tp_cost=0.0, tn_cost=0.0, fp_cost=0.0, fn_cost=0.0, baseline='zero_one')

Compute the predicted positive rate that minimizes the expected cost.

Examples

import numpy as np
from empulse.metrics import expected_savings_score

y_pred = [0.4, 0.8, 0.75, 0.1]
y_true = [0, 1, 1, 0]
fp_cost = np.array([4, 1, 2, 2])
fn_cost = np.array([1, 3, 3, 1])
expected_savings_score(y_true, y_pred, fp_cost=fp_cost, fn_cost=fn_cost)
On this page