mitao-cat / icml24_mcdp

Code for the ICML'24 paper "On the Maximal Local Disparity of Fairness-Aware Classifiers"

MCDP

This is the official implementation of our paper accepted by ICML'24:

On the Maximal Local Disparity of Fairness-Aware Classifiers

Jinqiu Jin, Haoxuan Li, Fuli Feng

Overview

We propose a novel fairness metric called MCDP for measuring the maximal local disparity of the fairness-aware classifiers. To accurately and efficiently calculate the MCDP, we develop a provably exact and an approximate calculation algorithm that greatly reduces the computational complexity with low estimation error. We further propose a bi-level optimization algorithm using a differentiable approximation of the MCDP for improving the algorithmic fairness. Extensive experiments on both tabular and image datasets validate that our fair training algorithm can achieve superior fairness-accuracy trade-offs.

Requirements

numpy==1.24.3
pandas==1.4.2
python==3.8.19
pytorch==1.11.0+cu113
scikit-learn==1.0.2
statsmodels==0.14.0
tabulate==0.9.0
torchvision==0.12.0+cu113

Data Preparation

Please refer to this link to download datasets and place them under the ./datasets directory. We have provided which files are used in the following structure (please kindly substitute the downloaded files to the empty files):

.
├── adult
│   └── raw
│       ├── adult.data
│       ├── adult.names
│       └── adult.test
├── bank_marketing
│   └── raw
│       └── bank-additional-full.csv
└── celeba
    └── raw
        └── celeba.csv

Run the Code

Training and Evaluating on a Specific Argument

For example, if you want to reproduce the result of the Adult (CelebA-A) dataset of DiffMCDP algorithm with temperature $\tau=20$, regularization strength $\lambda=0.2$, and seed 3, you can run the following commands:

cd ./code
python tabular_diffabcc.py --dataset adult --sensitive_attr sex --target_attr income --temperature 20 --lam 0.2 --seed 3
python image_advdebias.py --sensitive_attr Gender --target_attr Attractive --temperature 20 --lam 0.2 --seed 3

Reproducing All Results About the Learning Algorithms (DiffMCDP)

You can run the run.sh script as follows:

cd ./code
bash run.sh

This may generate experimental results supporting Figures 3 and 4, Tables 1 and 2. The intermediate data will be saved in the subdirectories of ./datasets as pickle and numpy files.

Calculation Time Comparison of the Exact and Approximate Algorithms

To obtain the raw experimental results of Figure 5, you can run python time_err.py (after reproducing results in the former step).

Acknowledgement

The implementation is based on the open-source fair machine learning framework FFB. We sincerely appreciate for their contribution to the trustworthy ML community!

Citation

@inproceedings{jin2024mcdp,
  title={On the Maximal Local Disparity of Fairness-Aware Classifiers},
  author={Jin, Jinqiu and Li, Haoxuan and Feng, Fuli},
  booktitle={International Conference on Machine Learning},
  year={2024},
  organization={PMLR}
}