MixtureMetric#

class empulse.metrics.MixtureMetric(components, defaults=None)[source]#

A weighted linear combination (“mixture”) of Metric objects.

Some domain metrics assume that an uncertain cost-matrix parameter follows a distribution that mixes point masses and/or continuous pieces. For example, EMP for Credit Scoring assumes the fraction of a defaulted loan that is recovered is 0 with probability p0 (full recovery), 1 with probability p1 (full loss), and otherwise follows a Uniform(0, 1) distribution. sympy.stats has no way to represent such a “spike + spike + continuous” random variable as a single object.

MixtureMetric sidesteps this by exploiting linearity of expectation. MaxProfit (and the other strategies) compute a linear functional of the assumed density of the uncertain parameter: the score, its gradient and Hessian with respect to model scores, and the optimal predicted-positive rate are all integrals (or derivatives of integrals) against that density. Since integration and differentiation are both linear, a mixture density’s value for any of these quantities is exactly the weight-averaged sum of each mixture component’s own value, evaluated independently. No approximation is involved, as long as each component’s own Metric already handles its piece of the mixture: a point mass is a deterministic Metric evaluated with the parameter fixed to that point, and a continuous piece is a stochastic Metric with that random variable.

optimal_threshold is the one exception: a threshold is a non-linear function of a rate (the score value at that rank in y_score), so it does not commute with the mixture’s linear weighting the way score and rate do. MixtureMetric computes it correctly by first combining the components’ rates and then converting that single combined rate to a threshold, rather than combining the components’ own thresholds.

Read more in the User Guide.

Parameters:

componentsSequence[MixtureComponent]: The components making up the mixture. Every component’s metric should share the same interpretation of y_true and y_score, and should be built from the same underlying cost-matrix pattern, only differing in how the uncertain symbol is fixed or distributed for that component.
defaultsMapping[str, float], optional: Default values for parameters (including weight parameters named by a component’s weight), used when not supplied at call time. Mirrors set_default for a plain Metric.

Examples

Reimplementing the cost structure behind empcs_score using MixtureMetric.

import sympy as sp
from empulse.metrics import CostMatrix, MaxProfit, Metric, MixtureComponent, MixtureMetric

gamma, roi = sp.symbols('gamma roi')
credit_matrix = CostMatrix().add_tp_benefit(gamma).add_fp_cost(roi)
metric_det = Metric(credit_matrix, MaxProfit())

gamma_rv = sp.stats.Uniform('gamma', 0, 1)
credit_matrix_stoch = CostMatrix().add_tp_benefit(gamma_rv).add_fp_cost(roi)
metric_stoch = Metric(credit_matrix_stoch, MaxProfit())

empcs_metric = MixtureMetric([
    MixtureComponent('success_rate', metric_det, {'gamma': 0.0}),
    MixtureComponent('default_rate', metric_det, {'gamma': 1.0}),
    MixtureComponent(
        lambda p: 1 - p['success_rate'] - p['default_rate'], metric_stoch, {}
    ),
])

y_true = [1, 0, 1, 0, 1]
y_proba = [0.9, 0.1, 0.8, 0.2, 0.7]
empcs_metric(y_true, y_proba, success_rate=0.55, default_rate=0.1, roi=0.2644)

__call__(y_true, y_score, **parameters)[source]#

Compute the weighted sum of each component’s metric score.

Parameters:

y_truearray-like of shape (n_samples,): The ground truth labels.
y_scorearray-like of shape (n_samples,): The predicted labels, probabilities, or decision scores.
parametersfloat or array-like of shape (n_samples,): Parameter values, including any weight parameters named by a component’s weight.

Returns:

scorefloat: The mixture’s combined score.

property direction#: The optimization direction shared by all components.

optimal_rate(y_true, y_score, **parameters)[source]#

Compute the weighted sum of each component’s optimal predicted positive rate.

Parameters:

y_truearray-like of shape (n_samples,): The ground truth labels.
y_scorearray-like of shape (n_samples,): The predicted labels, probabilities, or decision scores.
parametersfloat or array-like of shape (n_samples,): Parameter values, including any weight parameters named by a component’s weight.

Returns:

optimal_ratefloat: The mixture’s combined optimal predicted positive rate.

optimal_threshold(y_true, y_score, **parameters)[source]#

Compute the classification threshold that achieves the mixture’s combined optimal rate.

This is not the weighted sum of each component’s own optimal threshold: a threshold is a non-linear function of a rate (the score value at that rank), so it does not commute with the mixture’s linear weighting the way score and rate do. Instead, the combined optimal_rate is computed first, and the threshold that achieves that rate on the pooled y_score is returned, exactly mirroring how MaxProfit computes its own optimal threshold from its optimal rate.

Parameters:

y_truearray-like of shape (n_samples,): The ground truth labels.
y_scorearray-like of shape (n_samples,): The predicted labels, probabilities, or decision scores.
parametersfloat or array-like of shape (n_samples,): Parameter values, including any weight parameters named by a component’s weight.

Returns:

optimal_thresholdfloat | FloatNDArray: The optimal classification threshold(s).

property strategy#

A representative strategy shared by all components.

Raises a ValueError if components use different MetricStrategy types (e.g. mixing a MaxProfit component with a Cost component). This lets model code that inspects loss.strategy (e.g. to check isinstance(loss.strategy, MaxProfit)) work transparently with a MixtureMetric, exactly as it would with a plain Metric.