fairscoring.metrics.base ======================== .. py:module:: fairscoring.metrics.base .. autoapi-nested-parse:: Base class for bias metrics Classes ------- .. autoapisummary:: fairscoring.metrics.base.BaseBiasMetric fairscoring.metrics.base.BiasResult fairscoring.metrics.base.TwoGroupMetric fairscoring.metrics.base.TwoGroupBiasResult Module Contents --------------- .. py:class:: BaseBiasMetric(name, score_transform = None) Bases: :py:obj:`abc.ABC` Base bias metric. This class covers the basic bias computation workflow. :param name: Name of the metric. :type name: str :param score_transform: A transformation of the scores prior to the bias computation. There are two supported methods: - rescaling (to the interval `[0,1]`. In this case, the :meth:`~fairscoring.metrics._base.BaseBiasMetric.bias` method can take min and max scores. - quantile transformation. This leads to standardized bias measures. :type score_transform: {"rescale","quantile",None} .. py:method:: bias(scores, target, attribute, groups, favorable_target, *, min_score = None, max_score = None, n_permute = None, seed = None, prefer_high_scores = True) Bias computation :param scores: A list of scores :type scores: ArrayLike :param target: The binary target values. Must have the same length as `scores`. :type target: ArrayLike :param attribute: The protected attribute. Must have the same length as `scores`. :type attribute: ndarray :param groups: A list of groups. Each group is given by a value of the protected attribute. A value of `None` is used to define a group with all elements that are not in another group. :type groups: list :param favorable_target: The favorable outcome :type favorable_target: str or int :param min_score: The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling. :type min_score: float :param max_score: The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling :type max_score: float :param n_permute: Number of iterations for the permutation test. Permutation tests are only performed if this value is `>0`. :type n_permute: int, optional :param prefer_high_scores: Specify whether high scores or low scores are favorable. :type prefer_high_scores: bool, optional :returns: **bias** -- The computed bias (including intermediate results) :rtype: BiasResult :param seed: Random seed for the permutation test. Only required if the result need to be 100% reproducible. :type seed: int, optional .. py:method:: __call__(scores, target, attribute, groups, favorable_target, *, min_score = None, max_score = None, prefer_high_scores = True) Bias computation. This method allows to use the bias metric as a function. :param scores: A list of scores :type scores: ArrayLike :param target: The binary target values. Must have the same length as `scores`. :type target: ArrayLike :param attribute: The protected attribute. Must have the same length as `scores`. :type attribute: ndarray :param groups: A list of groups. Each group is given by a value of the protected attribute. A value of `None` is used to define a group with all elements that are not in another group. :type groups: list :param favorable_target: The favorable outcome :type favorable_target: str or int :param min_score: The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling. :type min_score: float :param max_score: The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling :type max_score: float :param prefer_high_scores: Specify whether high scores or low scores are favorable. :type prefer_high_scores: bool, optional :returns: **bias** -- The computed bias. :rtype: float .. rubric:: Notes This method offers fewer parameters than :meth:`~fairscoring.metrics._base.BaseBiasMetric.bias`, because not all will affect the pure bias value. .. py:class:: BiasResult(bias) A base class to store bias results. Child classes of this class are used to store metric-specific intermediate results that allow for further bias analyses, such as metric-specific plots. :param bias: The bias value :type bias: float :ivar bias: The bias value :vartype bias: float :ivar p_value: The p_value of statistical tests. :vartype p_value: float, optional .. py:class:: TwoGroupMetric(name, score_transform = None) Bases: :py:obj:`BaseBiasMetric` A mixin for metrics that only support two groups .. py:method:: bias(scores, target, attribute, groups, favorable_target, *, min_score = None, max_score = None, n_permute = None, seed = None, prefer_high_scores = True) Bias computation :param scores: A list of scores :type scores: ArrayLike :param target: The binary target values. Must have the same length as `scores`. :type target: ArrayLike :param attribute: The protected attribute. Must have the same length as `scores`. :type attribute: ndarray :param groups: A list of groups. Each group is given by a value of the protected attribute. A value of `None` is used to define a group with all elements that are not in another group. :type groups: list :param favorable_target: The favorable outcome :type favorable_target: str or int :param min_score: The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling. :type min_score: float :param max_score: The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling :type max_score: float :param n_permute: Number of iterations for the permutation test. Permutation tests are only performed if this value is `>0`. :type n_permute: int, optional :param prefer_high_scores: Specify whether high scores or low scores are favorable. :type prefer_high_scores: bool, optional :returns: **bias** -- The computed bias (including intermediate results) :rtype: BiasResult :param seed: Random seed for the permutation test. Only required if the result need to be 100% reproducible. :type seed: int, optional .. py:method:: __call__(scores, target, attribute, groups, favorable_target, *, min_score = None, max_score = None, prefer_high_scores = True) Bias computation. This method allows to use the bias metric as a function. :param scores: A list of scores :type scores: ArrayLike :param target: The binary target values. Must have the same length as `scores`. :type target: ArrayLike :param attribute: The protected attribute. Must have the same length as `scores`. :type attribute: ndarray :param groups: A list of groups. Each group is given by a value of the protected attribute. A value of `None` is used to define a group with all elements that are not in another group. :type groups: list :param favorable_target: The favorable outcome :type favorable_target: str or int :param min_score: The minimal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling. :type min_score: float :param max_score: The maximal score. This might influence the bias computation, e.g. by defining the integral bounds. This is also used for rescaling :type max_score: float :param prefer_high_scores: Specify whether high scores or low scores are favorable. :type prefer_high_scores: bool, optional :returns: **bias** -- The computed bias. :rtype: float .. rubric:: Notes This method offers fewer parameters than :meth:`~fairscoring.metrics._base.BaseBiasMetric.bias`, because not all will affect the pure bias value. .. py:class:: TwoGroupBiasResult(bias, pos, neg) Bases: :py:obj:`BiasResult` A base class to store bias results. Child classes of this class are used to store metric-specific intermediate results that allow for further bias analyses, such as metric-specific plots. :param bias: The bias value :type bias: float :param pos: The positive component of the bias :type pos: float :param neg: The negative component of the bias :type neg: float :ivar bias: The bias value :vartype bias: float :ivar pos: The positive component of the bias :vartype pos: float :ivar neg: The negative component of the bias :vartype neg: float .. py:property:: pos_component Proportion of the positive component in the total bias :rtype: Proportion of the positive component in the total bias .. py:property:: neg_component Proportion of the negative component in the total bias :rtype: Proportion of the negative component in the total bias