empulse.metrics

The metrics module contains a collection of metrics for evaluating the performance of models in the context of customer churn, credit scoring, and acquisition.

Build your own cost-sensitive metric

Metrics can be built by combining a CostMatrix with a MetricStrategy. The cost matrix defines the costs and benefits associated with different outcomes, while the metric strategy defines how to compute the metric based on the cost matrix and the model’s predictions.

Cost Matrix

CostMatrix

Class to create a custom value/cost-sensitive cost matrix.

Metric Strategies

MetricStrategy	Abstract base class for metric strategies.
Capability	A capability a MetricStrategy may or may not support.
MaxProfit	Strategy for the Expected Maximum Profit (EMP) metric.
MinCost	Strategy for the Minimum Cost metric.
EmpiricalMaxProfit	Strategy for the (empirical) maximum profit found by ranking samples by predicted score.
EmpiricalMinCost	Strategy for the Empirical Minimum Cost metric.
AUEPC	Strategy for the Area Under the Expected Profit Curve (AUEPC) metric.
Cost	Strategy for the Expected Cost metric.
Profit	Strategy for the Expected Profit metric.
LogCost	Strategy for the Expected Log Cost metric.
Savings	Strategy for the Expected Savings metric.

Metrics

BaseMetric	Abstract interface shared by every metric usable as a cost-sensitive loss.
Metric	Class to create a custom value/cost-sensitive metric.
MixtureComponent	One term of a MixtureMetric.
MixtureMetric	A weighted linear combination ("mixture") of Metric objects.

Extension Points

Abstract base classes for writing a custom MetricStrategy. Most users never need these.

LogitObjective	Class to compute the loss and gradient of a logistic regression objective.
ElasticNetPenalty	Elastic-net penalty on the non-intercept coefficients of a logit objective.
objective_scale_from_costs	Compute the natural magnitude of a cost matrix's per-sample training signal.

General Metrics

max_profit_score	Maximum Profit Measure (MP).
lift_score	Compute the lift score for the top fraction of predictions.
cost_loss	Cost of a classifier.
expected_cost_loss	Expected cost of a classifier.
expected_log_cost_loss	Expected log cost of a classifier.
savings_score	Cost savings of a classifier compared to using a baseline.
expected_savings_score	Expected savings of a classifier compared to a baseline.

Customer Acquisition Metrics

See Customer Acquisition Metrics for an explanation of the cost-benefit matrix.

empa_score	Expected Maximum Profit measure for customer Acquisition (EMPA).
mpa_score	Maximum Profit measure for customer Acquisition (MPA).
expected_cost_loss_acquisition	Expected cost of a classifier for customer acquisition.

Customer Churn Metrics

See Customer Churn Metrics for an explanation of the cost-benefit matrix.

empc_score	Expected Maximum Profit Measure for Customer Churn (EMPC).
mpc_score	Maximum Profit Measure for Customer Churn (MPC).
empb_score	Expected Maximum Profit Measure for B2B Customer Churn (EMPB).
auepc_score	Area Under the Expected Profit Curve (AUEPC).
expected_cost_loss_churn	Expected cost of a classifier for customer churn.

Credit Scoring Metrics

See Credit Scoring Metrics for an explanation of the cost-benefit matrix.

empcs_score	Expected Maximum Profit measure for Credit Scoring (EMPCS).
mpcs_score	Maximum Profit measure for Credit Scoring (MPCS).

Helper Functions

classification_threshold

Return classification threshold for given customer threshold.