ProfMEMPMClassifier#

class empulse.models.ProfMEMPMClassifier(*, tp_cost=0.0, tn_cost=0.0, fn_cost=0.0, fp_cost=0.0, loss=None, penalty='l2', lambda_reg=0.0, ridge_penalty=1e-06)[source]#

Profit-driven minimax probability machine classifier.

Learns a linear decision boundary that maximizes the worst-case (distribution-free) expected profit, using only the empirical means and covariances of each class through the multivariate Chebyshev-Cantelli inequality.

Setting lambda_reg=0 (default) reproduces the original Profit Maximizing Minimax Probability Machine (MEMPM): the weight vector is constrained to unit norm (||w||=1) and no additional regularization is applied.

Setting lambda_reg>0 switches to the Lp-regularized variant (Lp-ProfMEMPM): the unit-norm constraint is dropped and an L1 or L2 penalty (controlled by penalty) on the weight vector is added to the objective instead, controlling the scale of w.

Read more in the User Guide.

Parameters:

tp_costfloat or array-like, shape=(n_samples,), default=0.0

Cost of true positives. If float, then all true positives have the same cost. If array-like, then it is the cost of each true positive classification. Is overwritten if another tp_cost is passed to the fit method.

Note

It is not recommended to pass instance-dependent costs to the __init__ method. Instead, pass them to the fit method.

Note

Since this model only supports class-dependent costs, array-like costs are aggregated to their mean value before fitting.

tn_costfloat or array-like, shape=(n_samples,), default=0.0

Cost of true negatives. If float, then all true negatives have the same cost. If array-like, then it is the cost of each true negative classification. Is overwritten if another tn_cost is passed to the fit method.

Note

It is not recommended to pass instance-dependent costs to the __init__ method. Instead, pass them to the fit method.

Note

Since this model only supports class-dependent costs, array-like costs are aggregated to their mean value before fitting.

fn_costfloat or array-like, shape=(n_samples,), default=0.0

Cost of false negatives. If float, then all false negatives have the same cost. If array-like, then it is the cost of each false negative classification. Is overwritten if another fn_cost is passed to the fit method.

Note

It is not recommended to pass instance-dependent costs to the __init__ method. Instead, pass them to the fit method.

Note

Since this model only supports class-dependent costs, array-like costs are aggregated to their mean value before fitting.

fp_costfloat or array-like, shape=(n_samples,), default=0.0

Cost of false positives. If float, then all false positives have the same cost. If array-like, then it is the cost of each false positive classification. Is overwritten if another fp_cost is passed to the fit method.

Note

It is not recommended to pass instance-dependent costs to the __init__ method. Instead, pass them to the fit method.

Note

Since this model only supports class-dependent costs, array-like costs are aggregated to their mean value before fitting.

lossempulse.metrics.BaseMetric or None, default=None

Only BaseMetric instances built with the MaxProfit strategy are supported, since this model requires the costs and benefits to be reducible to four scalar values.

Note

If the costs or benefits contain stochastic variables, they are replaced by their mean/expectation before fitting.

If BaseMetric, metric parameters are passed as loss_params to the fit method.

If None, the loss is set to the Maximum Profit score.

penalty‘l1’ or ‘l2’, default=’l2’

Norm used in the regularization term. Only used when lambda_reg > 0.

lambda_regfloat, default=0.0

Regularization strength of the penalty term. Must be non-negative.

If 0.0, no regularization is applied and w is instead constrained to unit norm, reproducing the original (non-regularized) MEMPM formulation.

If greater than 0.0, the unit-norm constraint is dropped and w is regularized instead, reproducing the Lp-ProfMEMPM formulation.

ridge_penaltyfloat, default=1e-6

Small positive value added to the diagonal of the empirical covariance matrices to keep them positive definite (Tikhonov/ridge stabilization). This is independent of lambda_reg and is always applied.

Attributes:

classes_numpy.ndarray: Unique classes in the target found during fit.
coef_numpy.ndarray, shape=(n_features,): Coefficients of the linear decision boundary.
intercept_float: Intercept of the linear decision boundary.
result_scipy.optimize.OptimizeResult: Optimization result.

References

[1]

Bravo, C., & Vanderschueren, T. (2023, September). Profit maximizing distribution-free classifiers: a study on the minimax probability machine. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases.

Examples

from empulse.models import ProfMEMPMClassifier
from sklearn.datasets import make_classification

X, y = make_classification(n_features=4)

model = ProfMEMPMClassifier()
model.fit(X, y, tp_cost=-200, fp_cost=10)

fit(X, y, *, tp_cost=Parameter.UNCHANGED, fp_cost=Parameter.UNCHANGED, tn_cost=Parameter.UNCHANGED, fn_cost=Parameter.UNCHANGED, **loss_params)#

Fit the model according to the given training data.

Parameters:

Xarray-like of shape (n_samples, n_features): Training data.
yarray-like of shape (n_samples,): Target values.
tp_costfloat or array-like, shape=(n_samples,), default=$UNCHANGED$: Cost of true positives. If float, then all true positives have the same cost. If array-like, then it is the cost of each true positive classification.
fp_costfloat or array-like, shape=(n_samples,), default=$UNCHANGED$: Cost of false positives. If float, then all false positives have the same cost. If array-like, then it is the cost of each false positive classification.
tn_costfloat or array-like, shape=(n_samples,), default=$UNCHANGED$: Cost of true negatives. If float, then all true negatives have the same cost. If array-like, then it is the cost of each true negative classification.
fn_costfloat or array-like, shape=(n_samples,), default=$UNCHANGED$: Cost of false negatives. If float, then all false negatives have the same cost. If array-like, then it is the cost of each false negative classification.
loss_paramsAny: Additional parameter to be passed to the loss function.

Returns:

self: Fitted estimator.

get_metadata_routing()#

Get metadata routing of this object.

Please check User Guide on how the routing mechanism works.

Returns:

routingMetadataRequest: A MetadataRequest encapsulating routing information.

get_params(deep=True)#

Get parameters for this estimator.

Parameters:

deepbool, default=True: If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

paramsdict: Parameter names mapped to their values.

predict(X)#

Predict class labels for samples in X.

Parameters:

Xarray-like of shape (n_samples, n_features): Features.

Returns:

y_predndarray of shape (n_samples,): Predicted labels for each sample.

predict_proba(X)#

Compute predicted probabilities.

Parameters:

X2D array-like, shape=(n_samples, n_features): Features.

Returns:

y_pred2D numpy.ndarray, shape=(n_samples, 2): Predicted probabilities.

score(X, y, sample_weight=None)#

Return accuracy on provided data and labels.

In multi-label classification, this is the subset accuracy which is a harsh metric since you require for each sample that each label set be correctly predicted.

Parameters:

Xarray-like of shape (n_samples, n_features): Test samples.
yarray-like of shape (n_samples,) or (n_samples, n_outputs): True labels for X.
sample_weightarray-like of shape (n_samples,), default=None: Sample weights.

Returns:

scorefloat: Mean accuracy of self.predict(X) w.r.t. y.

set_fit_request(*, fn_cost='$UNCHANGED$', fp_cost='$UNCHANGED$', tn_cost='$UNCHANGED$', tp_cost='$UNCHANGED$')#

Configure whether metadata should be requested to be passed to the fit method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to fit if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to fit.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

fn_coststr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for fn_cost parameter in fit.
fp_coststr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for fp_cost parameter in fit.
tn_coststr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for tn_cost parameter in fit.
tp_coststr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for tp_cost parameter in fit.

Returns:

selfobject: The updated object.

set_params(**params)#

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as Pipeline). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Parameters:

**paramsdict: Estimator parameters.

Returns:

selfestimator instance: Estimator instance.

set_score_request(*, sample_weight='$UNCHANGED$')#

Configure whether metadata should be requested to be passed to the score method.

Note that this method is only relevant when this estimator is used as a sub-estimator within a meta-estimator and metadata routing is enabled with enable_metadata_routing=True (see sklearn.set_config). Please check the User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to score if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to score.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

Added in version 1.3.

Parameters:

sample_weightstr, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED: Metadata routing for sample_weight parameter in score.

Returns:

selfobject: The updated object.