AUEPC

class empulse.metrics.AUEPC(normalize=True)

Strategy for the Area Under the Expected Profit Curve (AUEPC) metric.

AUEPC ranks samples by their predicted score and tracks the cumulative profit of targeting (predicting positive for) the top-ranked fraction of samples, using the same benefits/costs as the other strategies (Cost, MaxProfit, …). This cumulative-profit curve is then compared against the curve of an oracle ranking that maximizes cumulative profit at every targeted fraction – the ratio of the two, integrated over the targeted fraction, is the AUEPC score. A perfect model (whose ranking matches the oracle) scores 1.0 (when normalize=True); an unhelpful/random ranking scores lower.

Unlike Cost and MaxProfit, AUEPC does not support use as a model training objective (no logit_objective or gradient_boost_objective): it evaluates a full ranking, not the outcome of a single sample.

Parameters:

normalizebool, default=True

Whether to normalize the AUEPC score so that a perfect model scores 1.0. This is only useful when part of the expected profit curve is negative.

.. seealso::

auepc_score : the underlying metric function.

build(tp_benefit, tn_benefit, fp_cost, fn_cost)

Build the metric strategy.

gradient_boost_objective(y_true, y_score, **parameters)

Compute the gradient of the metric with respect to gradient boosting instances.

Parameters:

y_true: array-like of shape (n_samples,)

The ground truth labels.

y_score: array-like of shape (n_samples,)

The predicted labels, probabilities, or decision scores (based on the chosen metric).

parameters: float or array-like of shape (n_samples,)

The parameter values for the costs and benefits defined in the metric. If any parameter is a stochastic variable, you should pass values for their distribution parameters. You can set the parameter values for either the symbol names or their aliases.

• If float, the same value is used for all samples (class-dependent).

• If array-like, the values are used for each sample (instance-dependent).

Returns:

gradientNDArray of shape (n_samples,)

The gradient of the metric loss with respect to the gradient boosting weights.

hessianNDArray of shape (n_samples,)

The hessian of the metric loss with respect to the gradient boosting weights.

logit_objective(features, y_true, C, l1_ratio, soft_threshold, fit_intercept, **parameters)

Compute the logit loss and its gradient with respect to the logistic regression weights.

Parameters:

featuresNDArray of shape (n_samples, n_features)

The features of the samples.

y_trueNDArray of shape (n_samples,)

The ground truth labels.

Cfloat

Regularization strength parameter. Smaller values specify stronger regularization.

l1_ratiofloat

The Elastic-Net mixing parameter, with range 0 <= l1_ratio <= 1. l1_ratio=0 corresponds to L2 penalty, l1_ratio=1 to L1 penalty.

soft_thresholdbool

Indicator of whether soft thresholding is applied during optimization.

fit_interceptbool

Specifies if an intercept should be included in the model.

parametersfloat or NDArray of shape (n_samples,)

The parameter values for the costs and benefits defined in the metric. If any parameter is a stochastic variable, you should pass values for their distribution parameters. You can set the parameter values for either the symbol names or their aliases.

• If float, the same value is used for all samples (class-dependent).

• If array-like, the values are used for each sample (instance-dependent).

Returns:

logistic_objectiveLogitObjective

A class that implements the logit loss and its gradient.

optimal_rate(y_true, y_score, **parameters)

Compute the predicted positive rate to optimize the metric value.

Parameters:

y_true: array-like of shape (n_samples,)

The ground truth labels.

y_score: array-like of shape (n_samples,)

The predicted labels, probabilities, or decision scores (based on the chosen metric).

parameters: float or array-like of shape (n_samples,)

The parameter values for the costs and benefits defined in the metric. If any parameter is a stochastic variable, you should pass values for their distribution parameters. You can set the parameter values for either the symbol names or their aliases.

• If float, the same value is used for all samples (class-dependent).

• If array-like, the values are used for each sample (instance-dependent).

Returns:

optimal_rate: float

The optimal predicted positive rate.

optimal_threshold(y_true, y_score, **parameters)

Compute the classification threshold(s) to optimize the metric value.

i.e., the score threshold at which an observation should be classified as positive to optimize the metric. For instance-dependent costs and benefits, this will return an array of thresholds, one for each sample. For class-dependent costs and benefits, this will return a single threshold value.

Parameters:

y_true: array-like of shape (n_samples,)

The ground truth labels.

y_score: array-like of shape (n_samples,)

The predicted labels, probabilities, or decision scores (based on the chosen metric).

parameters: float or array-like of shape (n_samples,)

The parameter values for the costs and benefits defined in the metric. If any parameter is a stochastic variable, you should pass values for their distribution parameters. You can set the parameter values for either the symbol names or their aliases.

• If float, the same value is used for all samples (class-dependent).

• If array-like, the values are used for each sample (instance-dependent).

Returns:

optimal_threshold: float | FloatNDArray

The optimal classification threshold(s).

prepare_boost_objective(y_true, **parameters)

Compute the gradient’s constant term of the metric wrt gradient boost.

Parameters:

y_trueNDArray of shape (n_samples,)

The ground truth labels.

parametersfloat or NDArray of shape (n_samples,)

The parameter values for the costs and benefits defined in the metric. If any parameter is a stochastic variable, you should pass values for their distribution parameters. You can set the parameter values for either the symbol names or their aliases.

• If float, the same value is used for all samples (class-dependent).

• If array-like, the values are used for each sample (instance-dependent).

Returns:

gradient_constNDArray of shape (n_samples, n_features)

The constant term of the gradient.

property requires_dynamic_boost_objective

Whether gradients must be recomputed from the metric each boosting round.

True for strategies whose per-sample loss is not linear in the predicted probability (e.g. LogCost) or that need the current round’s scores to locate a threshold (e.g. MaxProfit); such strategies cannot use the precomputed constant returned by prepare_boost_objective.

score(y_true, y_score, **parameters)

Compute the AUEPC score.

Parameters:

y_true: array-like of shape (n_samples,)

The ground truth labels.

y_score: array-like of shape (n_samples,)

The predicted labels, probabilities, or decision scores used to rank the samples.

parameters: float or array-like of shape (n_samples,)

The parameter values for the costs and benefits defined in the metric. If any parameter is a stochastic variable, you should pass values for their distribution parameters. You can set the parameter values for either the symbol names or their aliases.

• If float, the same value is used for all samples (class-dependent).

• If array-like, the values are used for each sample (instance-dependent).

Returns:

score: float

The AUEPC score.

to_latex(tp_benefit, tn_benefit, fp_cost, fn_cost)

Return the LaTeX representation of the metric.