kervias / dcd

[NeurIPS'23] Disentangling Cognitive Diagnosis with Limited Exercise Labels

[NeurIPS 2023] Disentangling Cognitive Diagnosis with Limited Exercise Labels

Official implementation of "Disentangling Cognitive Diagnosis with Limited Exercise Labels", NeurIPS 2023.

In this paper, we propose the Disentanglement based Cognitive Diagnosis (DCD) model to address the challenge of limited exercise labels in cognitive diagnosis. Specifically, we utilize students’ response records to model student proficiency, exercise difficulty and exercise label distribution. Then, we introduce two novel modules - group-based disentanglement and limited-labeled alignment modules - to disentangle the factors relevant to concepts and align them with real limited labels. Particularly, we introduce the tree-like structure of concepts with negligible cost for group-based disentangling, as concepts of different levels exhibit different independence relationships. Extensive experiments on widely used benchmarks demonstrate the superiority of our proposed model.

Install

The current implementation is based on EduStudio library that is a Unified and Templatized Framework for Student Assessment Models including Cognitive Diagnosis(CD) and Knowledge Tracing(KT).

1. Install pytorch==1.12.1 according to pytorch official site

2. Install EduStudio=1.0.0 according to edustudio github project

Run

1. run DCD

   python src/run.py  --cfg_file_name 20%_wo_fill.yaml
   python src/run.py  --cfg_file_name 10%_wo_fill.yaml
   python src/run.py  --cfg_file_name 100%_wo_fill.yaml --modeltpl_cfg.lambda_q 15.0 --modeltpl_cfg.g_beta_user 0.5 --modeltpl_cfg.g_beta_item 0.5 
   

2. run interpretable baselines, such as KaNCD

   python src/run_interpretable_baselines.py --modeltpl_cfg.cls KaNCD --cfg_file_name 10%_with_fill.yaml
   python src/run_interpretable_baselines.py --modeltpl_cfg.cls KaNCD --cfg_file_name 20%_with_fill.yaml
   python src/run_interpretable_baselines.py --modeltpl_cfg.cls KaNCD --cfg_file_name 100%_with_fill.yaml
   

3. run non-interpretable baselines, such as IRT

   python src/run_non_interpretable_baselines.py --modeltpl_cfg.cls IRT --cfg_file_name 10%_wo_fill.yaml
   python src/run_non_interpretable_baselines.py --modeltpl_cfg.cls IRT --cfg_file_name 20%_wo_fill.yaml
   python src/run_non_interpretable_baselines.py --modeltpl_cfg.cls IRT --cfg_file_name 100%_wo_fill.yaml
   

Citation

If you find this work useful in your research, please kindly cite our paper:

@inproceedings{chen2023disentangling,
 author = {Chen, Xiangzhi and Wu, Le and Liu, Fei and Chen, Lei and Zhang, Kun and Hong, Richang and Wang, Meng},
 booktitle = {Advances in Neural Information Processing Systems},
 editor = {A. Oh and T. Neumann and A. Globerson and K. Saenko and M. Hardt and S. Levine},
 pages = {18028--18045},
 publisher = {Curran Associates, Inc.},
 title = {Disentangling Cognitive Diagnosis with Limited Exercise Labels},
 url = {https://proceedings.neurips.cc/paper_files/paper/2023/file/3a14ae9951e8153a8fc814b5f506b5b7-Paper-Conference.pdf},
 volume = {36},
 year = {2023}
}