fairscoring.metrics.roc

fairscoring.metrics.roc#

ROC-Base metrics.

Classes#

ROCBiasMetric

ROC-Based Fairness Metrics.

Module Contents#

class fairscoring.metrics.roc.ROCBiasMetric(name, bias_type)#

Bases: fairscoring.metrics.base.TwoGroupMetric

ROC-Based Fairness Metrics.

These metrics compute the absolute between ROC area (ABROCA) of two roc-curve [VoBC21]. The following two metrics can be distinguished:

roc

Compares the classic roc-curves of two groups with each other.

xroc

Builds roc curves with class 0 samples from one group and class 1 samples from the other group.

This metric returns a TwoGroupBiasResult object, which allows to split the bias in positive and negative parts.

Parameters:

  • name (str) – Name of the metric.

  • bias_type ({"roc", "xroc"})

Notes

This metric is not influenced by monotonic score transformations. For this reason, no score_transform parameter is provided.

bias(scores, target, attribute, groups, favorable_target, *, min_score=None, max_score=None, n_permute=None, seed=None, prefer_high_scores=True)#

Bias computation

Parameters:

  • scores (ArrayLike) – A list of scores

  • target (ArrayLike) – The binary target values. Must have the same length as scores.

  • attribute (ndarray) – The protected attribute. Must have the same length as scores.

  • groups (list) – A list of groups. Each group is given by a value of the protected attribute. A value of None is used to define a group with all elements that are not in another group.

  • favorable_target (str or int) – The favorable outcome

  • min_score (float) – The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling.

  • max_score (float) – The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling

  • n_permute (int, optional) – Number of iterations for the permutation test. Permutation tests are only performed if this value is >0.

  • prefer_high_scores (bool, optional) – Specify whether high scores or low scores are favorable.

  • seed (int, optional) – Random seed for the permutation test. Only required if the result need to be 100% reproducible.

Returns:

bias – The computed bias (including intermediate results)

Return type:

BiasResult

__call__(scores, target, attribute, groups, favorable_target, *, min_score=None, max_score=None, prefer_high_scores=True)#

Bias computation.

This method allows to use the bias metric as a function.

Parameters:

  • scores (ArrayLike) – A list of scores

  • target (ArrayLike) – The binary target values. Must have the same length as scores.

  • attribute (ndarray) – The protected attribute. Must have the same length as scores.

  • groups (list) – A list of groups. Each group is given by a value of the protected attribute. A value of None is used to define a group with all elements that are not in another group.

  • favorable_target (str or int) – The favorable outcome

  • min_score (float) – The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling.

  • max_score (float) – The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling

  • prefer_high_scores (bool, optional) – Specify whether high scores or low scores are favorable.

Returns:

bias – The computed bias.

Return type:

float

Notes

This method offers fewer parameters than bias(), because not all will affect the pure bias value.

Contents