# Home

## 📍 Introduction

FairRankTune is a an open-source Python toolkit supporting end-to-end fair ranking workflows, analysis, auditing, and experimentation. FairRankTune provides researchers, practitioners, and educators with a self-contained module for generating ranked data, ranking strategies, and popular ranking-based fairness metrics.

For a quick overview, follow the Usage section.

For a in-depth overview, follow the Examples section.

## ✨ Features

### 🎨 Fairness-Aware Ranked Data Generation

`RankTune`

is a pseudo-stochastic data generation method for creating fairness-aware ranked lists using the fairness concept of statistical parity. Included in the `RankTune`

module, it creates ranking(s) based on the `phi`

representativeness parameter. When `phi = 0`

then the generated ranked list(s) does not represent groups fairly, and as `phi`

increases groups are represented more and more fairly; thus `phi = 1`

groups are fairly represented. RankTune uses a pseudo-random process to generate fairness-aware ranked data. RankTune can generate ranked data from user provided group sizes, from existing datasets, along with producing relevance scores accompanying the ranked list(s).

Please refer to the documentation for additional information.

### 📏 Metrics

`FairRankTune`

provides several metrics for evaluating the fairness of ranked lists in the `Metrics`

module. The table below provides a high-level overview of each metric. These metrics encompass a variety of fair ranking metrics, including both group and individual fairness, along with both score-based and statistical parity metrics.

Metric |
Abbreviation |
Fairness (Group or Individual) |
Score-based |
Statistical Parity |
Reference |
---|---|---|---|---|---|

Group Exposure | EXP | Group | No | Yes | Singh et al. |

Exposure Utility | EXPU | Group | Yes | No | Singh et al. |

Exposure Realized Utility | EXPRU | Group | Yes | No | Singh et al. |

Attention Weighted Rank Fairness | AWRF | Group | No | Yes | Sapiezynski et al. |

Exposure Rank Biased Precision Equality | ERBE | Group | No | No | Kirnap et al. |

Exposure Rank Biased Precision Proportionality | ERBP | Group | No | Yes | Kirnap et al. |

Exposure Rank Biased Precision Proportional to Relevance | ERBR | Group | Yes | No | Kirnap et al. |

Attribute Rank Parity | ARP | Group | No | Yes | Cachel et al. |

Normalized Discounted KL-Divergence | NDKL | Group | No | Yes | Geyik et al. |

Inequity of Amortized Attention | IAA | Individual | Yes | No | Biega et al. |

Please refer to the Metrics documentation for further details.

### ⚖️ Fair Ranking Methods

`FairRankTune`

provides several fair ranking algorithms in the `Rankers`

module. The DetConstSort and Epsilon-Greedy fair ranking algorithms can be used to re-rank a given ranking with the objective of making the resulting ranking fair.

Please refer to the documentation for further details.

## 🔌 Requirements

As of`v.0.0.6`

, FairRankTune requires `python>=3.8`

.
## 💾 Installation

## 💡 Usage

### 🎨 Fairness-Aware Ranked Data Generation

`RankTune`

can be used to generate ranking(s) from `group_proportions`

, a numpy array with each group's proportion of the total items, `num_items`

, by using the `GenFromGroups()`

function.

```
import FairRankTune as frt
import numpy as np
import pandas as pd
from FairRankTune import RankTune, Metrics
#Generate a biased (phi = 0.1) ranking of 1000 items, with four groups of 100,
# 200, 300, and 400 items each.
group_proportions = np.asarray([.1, .2, .3, .4]) #Array of group proportions
num_items = 1000 #1000 items to be in the generated ranking
phi = 0.1
r_cnt = 1 #Generate 1 ranking
seed = 10 #For reproducibility
ranking_df, item_group_dict = frt.RankTune.GenFromGroups(group_proportions,
num_items,phi, r_cnt, seed)
#Calculate EXP with a MinMaxRatio
EXP_minmax, avg_exposures_minmax = frt.Metrics.EXP(ranking_df, item_group_dict,
'MinMaxRatio')
print("EXP of generated ranking: ", EXP_minmax, "avg_exposures: ",
avg_exposures_minmax)
```

Output:

```
EXP of generated ranking: 0.511665941043515
avg_exposures: {0: 0.20498798214669187, 1: 0.13126425437156242, 2: 0.11461912123646827, 3: 0.10488536878769836}
```

`RankTune`

can be used to generate ranking(s) from `item_group_dict`

, a dictionary of items where the keys are each item's group by using the `GenFromItems()`

function.

```
import FairRankTune as frt
import numpy as np
import pandas as pd
from FairRankTune import RankTune, Metrics
#Generate a biased (phi = 0.1) ranking
item_group_dict = dict(Joe= "M", David= "M", Bella= "W", Heidi= "W",
Amy = "W", Jill= "W", Jane= "W", Dave= "M", Nancy= "W", Nick= "M")
phi = 0.1
r_cnt = 1 #Generate 1 ranking
seed = 10 #For reproducibility
ranking_df, item_group_dict = frt.RankTune.GenFromItems(item_group_dict, phi,
r_cnt, seed)
#Calculate EXP with a MinMaxRatio
EXP_minmax, avg_exposures_minmax = frt.Metrics.EXP(ranking_df,
item_group_dict, 'MinMaxRatio')
print("EXP of generated ranking: ", EXP_minmax,
"avg_exposures: ", avg_exposures_minmax)
```

Output:

```
EXP of generated ranking: 0.5158099476966725 avg_exposures: {'M': 0.6404015779112127, 'W': 0.33032550440724917}
```

For further detail on how to use `RankTune`

to generate relevance scores see the RankTune documentation.

### 📏 Metrics

The `Metric`

library can be used to assess the fairness of rankings. Our current offering contains 10 core metrics across individual and group fairness. Group fairness metrics include both metrics that incorporate relevance scores associated with items, so-called score-based fairness, and statistical parity metrics that are based on the representation of groups. A key functionality of the `Metrics`

library in `FairRankTune`

is providing toolkit users multiple choices for how to calculate a given top-level fairness metric. For instance, for group exposure, a popular fairness criteria, `Metrics`

offers seven ways of calculating a top-level exposure metric (e.g., min-max ratios, max absolute difference, L-2 norms of per-group exposures, etc.).

```
import FairRankTune as frt
import pandas as pd
import numpy as np
ranking_df = pd.DataFrame(["Joe", "Jack", "Nick", "David",
"Mark", "Josh", "Dave", "Bella", "Heidi", "Amy"])
item_group_dict = dict(Joe= "M", David= "M", Bella= "W",
Heidi= "W", Amy = "W", Mark= "M", Josh= "M", Dave= "M", Jack= "M", Nick= "M")
#Calculate EXP with a MaxMinDiff
EXP, avg_exposures = frt.Metrics.EXP(ranking_df, item_group_dict,
'MaxMinDiff')
print("EXP: ", EXP, "avg_exposures: ", avg_exposures)
```

### ⚖️ Fair Ranking Algorithms

```
import FairRankTune as frt
import numpy as np
import pandas as pd
from FairRankTune import RankTune, Metrics
import random
#Generate a biased (phi = 0) ranking of 1000 items, with two groups of
#100 and 900 items each.
group_proportions = np.asarray([.1, .9]) #Array of group proportions
num_items = 1000 #1000 items to be in the generated ranking
phi = 0 #Biased ranking
r_cnt = 1 #Generate 1 ranking
ranking_df, item_group_dict, scores_df = frt.RankTune.ScoredGenFromGroups(
group_proportions, num_items, phi, r_cnt, 'uniform', seed)
#Calculate EXP with a MinMaxRatio
EXP_minmax, avg_exposures_minmax = frt.Metrics.EXP(ranking_df,
item_group_dict, 'MinMaxRatio')
print("EXP before Epsilon-Greedy: ", EXP_minmax,
"avg_exposures before Epsilon-Greedy: ", avg_exposures_minmax)
#Rerank using Epsilon-Greedy
seed = 2 #For reproducibility
epsilon = .6
reranking_df, item_group_d, reranking_scores = frt.Rankers.EPSILONGREEDY(
ranking_df, item_group_dict, scores_df, epsilon, seed)
#Calculate EXP with a MinMaxRatio post Epsilon-Greedy
EXP, avg_exposures= frt.Metrics.EXP(reranking_df, item_group_d,
'MinMaxRatio')
print("EXP after Epsilon-Greedy: ", EXP,
"avg_exposures after Epsilon-Greedy: ", avg_exposures)
```

Output:

```
EXP before Epsilon-Greedy: 0.5420744267551784 avg_exposures before Epsilon-Greedy: {0: 0.2093867087428094, 1: 0.11350318011191189}
EXP after Epsilon-Greedy: 0.7689042373241246 avg_exposures after Epsilon-Greedy: {0: 0.15541589156986096, 1: 0.1194999375755728}
```

## 📖 Examples

Topic | Link |
---|---|

Quickstart | |

RankTune Overview | |

RankTune Augmenting Datasets | |

Statistical Parity Metrics | |

Score-based (Group & Individual) Metrics | |

Using Fair Ranking Algorithms |

## 📚 Documentation

Check out the documentation for more details and example notebooks.

## 🎓 Citation

If you end up using FairRankTune in your work, please consider citing it:

## BibTeX

## ⁉️ Feature Requests

We believe in open-source community driven software. Would you like to see other functionality implemented? Please, open a feature request. Is there a bug or issue ? Please, open a github issue.

## 👋 Want to contribute?

Would you like to contribute? Please, send me an e-mail.

## 📄 License

FairRankTune is open-sourced software licensed under the BSD-3-Clause license.