fairscoring.metrics.integral ============================ .. py:module:: fairscoring.metrics.integral .. autoapi-nested-parse:: Intergal bias metrics that measure the differences between cumulative distribution functions. Classes ------- .. autoapisummary:: fairscoring.metrics.integral.IntegralMetric fairscoring.metrics.integral.IntegralBiasResult fairscoring.metrics.integral.WassersteinMetric Module Contents --------------- .. py:class:: IntegralMetric(fairness_type, name, score_transform = None) Bases: :py:obj:`fairscoring.metrics.base.TwoGroupMetric` Base Class for Integral Metrics that compare cdfs. :param fairness_type: Specifies the type of fairness that is measured. Accepted values are: 1. `"IND"` (Independence), 2. `"EO"` (Equal Opportunity), 3. `"PE"` (Predictive Equality), :type fairness_type: {"IND", "EO", "PE"} :param name: Name of the Metric :type name: str :param score_transform: A transformation of the scores prior to the bias computation. There are two supported methods: - rescaling (to the interval `[0,1]`. In this case, the :meth:`~fairscoring.metrics._integral.IntegralMetric.bias` method can take min and max scores. - quantile transformation. This leads to standardized bias measures. :type score_transform: {"rescale","quantile",None} .. py:method:: bias(scores, target, attribute, groups, favorable_target, *, min_score = None, max_score = None, n_permute = None, seed = None, prefer_high_scores = True) Bias computation :param scores: A list of scores :type scores: ArrayLike :param target: The binary target values. Must have the same length as `scores`. :type target: ArrayLike :param attribute: The protected attribute. Must have the same length as `scores`. :type attribute: ndarray :param groups: A list of groups. Each group is given by a value of the protected attribute. A value of `None` is used to define a group with all elements that are not in another group. :type groups: list :param favorable_target: The favorable outcome :type favorable_target: str or int :param min_score: The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling. :type min_score: float :param max_score: The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling :type max_score: float :param n_permute: Number of iterations for the permutation test. Permutation tests are only performed if this value is `>0`. :type n_permute: int, optional :param prefer_high_scores: Specify whether high scores or low scores are favorable. :type prefer_high_scores: bool, optional :returns: **bias** -- The computed bias (including intermediate results) :rtype: BiasResult :param seed: Random seed for the permutation test. Only required if the result need to be 100% reproducible. :type seed: int, optional .. py:method:: __call__(scores, target, attribute, groups, favorable_target, *, min_score = None, max_score = None, prefer_high_scores = True) Bias computation. This method allows to use the bias metric as a function. :param scores: A list of scores :type scores: ArrayLike :param target: The binary target values. Must have the same length as `scores`. :type target: ArrayLike :param attribute: The protected attribute. Must have the same length as `scores`. :type attribute: ndarray :param groups: A list of groups. Each group is given by a value of the protected attribute. A value of `None` is used to define a group with all elements that are not in another group. :type groups: list :param favorable_target: The favorable outcome :type favorable_target: str or int :param min_score: The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling. :type min_score: float :param max_score: The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling :type max_score: float :param prefer_high_scores: Specify whether high scores or low scores are favorable. :type prefer_high_scores: bool, optional :returns: **bias** -- The computed bias. :rtype: float .. rubric:: Notes This method offers fewer parameters than :meth:`~fairscoring.metrics._base.BaseBiasMetric.bias`, because not all will affect the pure bias value. .. py:class:: IntegralBiasResult(bias, pos, neg, cdf_x, cdfs) Bases: :py:obj:`fairscoring.metrics.base.TwoGroupBiasResult` An extended bias result that also stores groupwise cumulative distribution functions (cdfs). :param bias: The bias value :type bias: float :param pos: The positive component of the bias :type pos: float :param neg: The negative component of the bias :type neg: float :param cdf_x: x-values at which the cdfs are stored. This array is 1-dimensional :type cdf_x: ArrayLike :param cdfs: A list of cdfs. :type cdfs: List of ArrayLike :ivar bias: The bias value :vartype bias: float :ivar pos: The positive component of the bias :vartype pos: float :ivar neg: The negative component of the bias :vartype neg: float :ivar cdf_x: x-values at which the cdfs are stored. This array is 1-dimensional :vartype cdf_x: ArrayLike :ivar cdfs: A list of cdfs. :vartype cdfs: List of ArrayLike .. py:property:: pos_component Proportion of the positive component in the total bias :rtype: Proportion of the positive component in the total bias .. py:property:: neg_component Proportion of the negative component in the total bias :rtype: Proportion of the negative component in the total bias .. py:class:: WassersteinMetric(fairness_type, name, score_transform = None, p = 1) Bases: :py:obj:`IntegralMetric` A metric that measures the differences between distributions using the Wasserstein Distance [BeDB24]_. This metric can be used to measure independence and separation bias. The `fairness_type`-parameter specifies which bias to measure and hence which distribution will be compared. This metric returns a :class:`~IntegralBiasResult` object, which allows to split the bias in positive and negative parts. Furthermore, it stores the cumulative distribution functions of the groups. :param fairness_type: Specifies the type of fairness that is measured. Accepted values are: 1. `"IND"` (Independence), 2. `"EO"` (Equal Opportunity), 3. `"PE"` (Predictive Equality), :type fairness_type: {"IND", "EO", "PE"} :param name: Name of the Metric :type name: str :param score_transform: A transformation of the scores prior to the bias computation. There are two supported methods: - rescaling (to the interval `[0,1]`. In this case, the :meth:`~fairscoring.metrics._integral.WassersteinMetric.bias` method can take min and max scores. - quantile transformation. This leads to standardized bias measures. :type score_transform: {"rescale","quantile",None} :param p: Exponent for the Wasserstein Distance. Use the default of 1 to get the Earthmover Distance :type p: float, default=1 .. py:method:: bias(scores, target, attribute, groups, favorable_target, *, min_score = None, max_score = None, n_permute = None, seed = None, prefer_high_scores = True) Bias computation :param scores: A list of scores :type scores: ArrayLike :param target: The binary target values. Must have the same length as `scores`. :type target: ArrayLike :param attribute: The protected attribute. Must have the same length as `scores`. :type attribute: ndarray :param groups: A list of groups. Each group is given by a value of the protected attribute. A value of `None` is used to define a group with all elements that are not in another group. :type groups: list :param favorable_target: The favorable outcome :type favorable_target: str or int :param min_score: The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling. :type min_score: float :param max_score: The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling :type max_score: float :param n_permute: Number of iterations for the permutation test. Permutation tests are only performed if this value is `>0`. :type n_permute: int, optional :param prefer_high_scores: Specify whether high scores or low scores are favorable. :type prefer_high_scores: bool, optional :returns: **bias** -- The computed bias (including intermediate results) :rtype: BiasResult :param seed: Random seed for the permutation test. Only required if the result need to be 100% reproducible. :type seed: int, optional .. py:method:: __call__(scores, target, attribute, groups, favorable_target, *, min_score = None, max_score = None, prefer_high_scores = True) Bias computation. This method allows to use the bias metric as a function. :param scores: A list of scores :type scores: ArrayLike :param target: The binary target values. Must have the same length as `scores`. :type target: ArrayLike :param attribute: The protected attribute. Must have the same length as `scores`. :type attribute: ndarray :param groups: A list of groups. Each group is given by a value of the protected attribute. A value of `None` is used to define a group with all elements that are not in another group. :type groups: list :param favorable_target: The favorable outcome :type favorable_target: str or int :param min_score: The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling. :type min_score: float :param max_score: The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling :type max_score: float :param prefer_high_scores: Specify whether high scores or low scores are favorable. :type prefer_high_scores: bool, optional :returns: **bias** -- The computed bias. :rtype: float .. rubric:: Notes This method offers fewer parameters than :meth:`~fairscoring.metrics._base.BaseBiasMetric.bias`, because not all will affect the pure bias value.