1.6. Credit Scoring Metrics#

In credit scoring the classifier decides which loan applications to reject. The two errors are very unlike each other: rejecting a good applicant forfeits the interest you would have earned, while accepting a bad one loses part of the principal. Empulse ships two ready-made metrics for this trade-off, following the profit-based framework of Verbraken et al. [1].

Note

These names are prebuilt Metric and MixtureMetric instances, not functions. Earlier versions of Empulse exposed hand-written empcs and mpcs functions returning a (score, threshold) tuple; those have been removed. Call the metric for the score, and use optimal_rate for the fraction of applicants to reject.

1.6.1. The Cost-Benefit Matrix#

Here the positive class is a defaulter, so a “predicted positive” means the application is rejected. Both quantities are expressed as fractions of the loan amount, which makes the metric scale-free:

  • \(\lambda\) (loan_lost_rate) — the fraction of the principal lost when a loan defaults. Correctly rejecting a defaulter saves you this.

  • \(ROI\) (roi) — the return you would have earned on a good loan. Wrongly rejecting a good applicant costs you this.

Actual defaulter \(y_i = 1\)

Actual non-defaulter \(y_i = 0\)

Rejected \(\hat{y}_i = 1\)

tp_benefit \(= \lambda\)

fp_cost \(= ROI\)

Accepted \(\hat{y}_i = 0\)

fn_cost \(= 0\)

tn_benefit \(= 0\)

1.6.2. Maximum Profit for Credit Scoring (MPCS)#

mpcs_score treats the loss given default as a single known fraction.

Defaults: loan_lost_rate=0.275, roi=0.2644.

import numpy as np
from empulse.metrics import mpcs_score

y_true = np.array([0, 1, 0, 1, 0, 1, 0, 1])
y_score = np.array([0.1, 0.9, 0.2, 0.8, 0.3, 0.7, 0.4, 0.6])

profit = mpcs_score(y_true, y_score)
profit_custom = mpcs_score(y_true, y_score, loan_lost_rate=0.4, roi=0.30)

1.6.3. Expected Maximum Profit for Credit Scoring (EMPCS)#

The loss given default varies from loan to loan, and lenders typically observe it as a mixture: some defaults are recovered in full, some are total write-offs, and the rest fall in between. empcs_score captures exactly that shape. It is a MixtureMetric combining three components:

  1. full recovery (\(\lambda = 0\)), weighted by success_rate

  2. total loss (\(\lambda = 1\)), weighted by default_rate

  3. a partial loss in between, taking the remaining weight

Defaults: success_rate=0.55, default_rate=0.1, roi=0.2644.

from empulse.metrics import empcs_score

expected_profit = empcs_score(y_true, y_score)
expected_profit_custom = empcs_score(y_true, y_score, success_rate=0.4, default_rate=0.2)

1.6.3.1. How many applicants should you reject?#

from empulse.metrics import classification_threshold

reject_fraction = empcs_score.optimal_rate(y_true, y_score)
threshold = classification_threshold(y_true, y_score, customer_threshold=reject_fraction)

1.6.4. Instance-dependent alternatives#

Both metrics above are class-dependent: every loan shares the same roi and loss rate. Real portfolios are not like that — a loan’s expected loss depends on its own principal and term.

For instance-dependent credit scoring, build a metric from the cost matrix that fetch_give_me_some_credit and load_credit_scoring_pakdd already ship, which expresses the false-negative cost as a per-applicant credit line times the loss given default.

import pandas as pd
from empulse.datasets import fetch_give_me_some_credit
from empulse.metrics import Metric, Savings

dataset = fetch_give_me_some_credit(backend=pd)

# replace with your own model's predicted probabilities
y_proba = np.random.default_rng(0).uniform(size=len(dataset.target))

savings = Metric(dataset.cost_matrix, Savings())
score = savings(dataset.target, y_proba, **dataset.instance_costs)

See Credit Risk Assessment on a Private Label Credit Card Application and 2011 Kaggle competition Give Me Some Credit for the full cost matrices, and Define your own cost-sensitive or value metric for writing your own from scratch.

1.6.5. See also#

1.6.6. References#