Methods for quantifying prevalence of underreported medical conditions like IPV.
All code was tested using Python 3.9.6. I recommend installing mamba to speed up installation, but you can substitute conda in just fine. Run the following commands from the root directory to install required dependencies (~5-10 minutes):
mamba env create --name purple --file=purple_environment.yml
source activate purple
git clone https://github.com/ML-KULeuven/SAR-PU
cat requirements.txt | xargs -n 1 pip install
cd SAR-PU
pip install -e sarpu
python make_km_lib.py
pip install -e lib/tice
pip install -e lib/km
Once you have run these commands, you will want to set paths for the figures, results, and models in relative_prevalence_benchmark/paths.py
.
⚡️ Quick option (2 minutes): Compare PURPLE to the baselines on a completely synthetic dataset in this notebook: relative_prevalence_benchmark/Minimal Example.ipynb
.
⏰ Slow option (~1 hour): Reproduce Figure 2 by running ./run_synthetic_expmts.sh
from the relative_prevalence_benchmark
directory.
- Intimate Partner Violence data.
- Download data from MIMIC-IV and MIMIC-IV ED. You will need to complete training in order to access both.
- Set paths for MIMIC-IV and MIMIC-IV ED in the file
./MIMIC_notebooks/mimic_paths.py
. - Generate each of the semi-synthetic dataset by running each of the following notebooks: "Generate Random Semi-Simulated Data.ipynb", "Generate Endometriosis Correlation Data.ipynb", "Generate IPV Semi-Simulated Data.ipynb", all under
./MIMIC_notebooks
. - Generate the real dataset by running the following notebook: "./MIMIC_notebooks/Generate Real IPV Data.ipynb".
- Content Moderation data.
- Download the pre-trained model, trained via ERM, from this link. Store "best_model.pth" in the
./WILDS_notebooks
folder. - Generate the CivilComments dataset by running the following notebook: "./WILDS_notebooks/Preprocess CivilComments Dataset.ipynb".
- Set global paths to the results, models, and figure directories in
relative_prevalence_benchmark/paths.py
.
- Download the pre-trained model, trained via ERM, from this link. Store "best_model.pth" in the
To reproduce experiments in the main text, run the following:
cd relative_prevalence_benchmark
./run_synthetic_expmts.sh
./run_semisynthetic_expmts.sh
./run_mimic_expmts.sh
./run_cmod_expmts.sh
To reproduce experiments in the supplement, run the following:
cd relative_prevalence_benchmark
./run_unconstrained_models.sh
./run_synthetic_cmod_expmts.sh
You can generate each of the figures and tables in the paper using the notebooks in ./relative_prevalence_benchmark/
; each notebook title includes the figure numbers it reproduces. A short guide to the files in ./relative_prevalence_benchmark
is that benchmark*.py
files implement a comparison of PURPLE to other baselines for a specific dataset, where different files differ by the datasets they use. All files of the form run*.sh
are experiment scripts, meant to fire off all comparisons for a particular experiment.
Email [email protected] if you run into any issues!