2.5. Credit Scoring Metrics#

In credit scoring the classifier decides which loan applications to reject. The two errors are very unlike each other: rejecting a good applicant forfeits the interest you would have earned, while accepting a bad one loses part of the principal. Empulse ships two ready-made metrics for this trade-off, following the profit-based framework of Verbraken et al. [1].

Note

These names are prebuilt Metric and MixtureMetric instances, not functions. Call one to get the score, and use optimal_rate for the fraction of applicants to reject.

2.5.1. The Cost-Benefit Matrix#

Here the positive class is a defaulter, so a “predicted positive” means the application is rejected. Both quantities are expressed as fractions of the loan amount, which makes the metric scale-free:

  • \(\lambda\) (loan_lost_rate) — the fraction of the principal lost when a loan defaults. Correctly rejecting a defaulter saves you this.

  • \(ROI\) (roi) — the return you would have earned on a good loan. Wrongly rejecting a good applicant costs you this.

Actual defaulter \(y_i = 1\)

Actual non-defaulter \(y_i = 0\)

Rejected \(\hat{y}_i = 1\)

tp_benefit \(= \lambda\)

fp_cost \(= ROI\)

Accepted \(\hat{y}_i = 0\)

fn_cost \(= 0\)

tn_benefit \(= 0\)

2.5.2. Maximum Profit for Credit Scoring (MPCS)#

mpcs_score treats the loss given default as a single known fraction.

Defaults: loan_lost_rate=0.275, roi=0.2644.

import numpy as np
from empulse.metrics import mpcs_score

y_true = np.array([0, 1, 0, 1, 0, 1, 0, 1])
y_score = np.array([0.1, 0.9, 0.2, 0.8, 0.3, 0.7, 0.4, 0.6])

profit = mpcs_score(y_true, y_score)
profit_custom = mpcs_score(y_true, y_score, loan_lost_rate=0.4, roi=0.30)

2.5.3. Expected Maximum Profit for Credit Scoring (EMPCS)#

The loss given default varies from loan to loan, and lenders typically observe it as a mixture: some defaults are recovered in full, some are total write-offs, and the rest fall in between. empcs_score captures exactly that shape. It is a MixtureMetric combining three components:

  1. full recovery (\(\lambda = 0\)), weighted by success_rate

  2. total loss (\(\lambda = 1\)), weighted by default_rate

  3. a partial loss in between, taking the remaining weight

Defaults: success_rate=0.55, default_rate=0.1, roi=0.2644.

from empulse.metrics import empcs_score

expected_profit = empcs_score(y_true, y_score)
expected_profit_custom = empcs_score(y_true, y_score, success_rate=0.4, default_rate=0.2)

2.5.3.1. How many applicants should you reject?#

from empulse.metrics import classification_threshold

reject_fraction = empcs_score.optimal_rate(y_true, y_score)
threshold = classification_threshold(y_true, y_score, customer_threshold=reject_fraction)

2.5.4. Instance-dependent alternatives#

Both metrics above are class-dependent: every loan shares the same roi and loss rate. Real portfolios are not like that — a loan’s expected loss depends on its own principal and term.

For instance-dependent credit scoring, build a metric from the cost matrix that fetch_give_me_some_credit and load_credit_scoring_pakdd already ship, which expresses the false-negative cost as a per-applicant credit line times the loss given default.

import pandas as pd
from empulse.datasets import fetch_give_me_some_credit
from empulse.metrics import Metric, Savings

dataset = fetch_give_me_some_credit(backend=pd)

# replace with your own model's predicted probabilities
y_proba = np.random.default_rng(0).uniform(size=len(dataset.target))

savings = Metric(dataset.cost_matrix, Savings())
score = savings(dataset.target, y_proba, **dataset.instance_costs)

See Credit Risk Assessment on a Private Label Credit Card Application and 2011 Kaggle competition Give Me Some Credit for the full cost matrices, and Worked cost matrices for writing your own from scratch.

2.5.5. See also#

2.5.6. References#