Data and code for paper titled New User Intent Discovery with Robust Pseudo Label Training and Source Domain Joint-training (IEEE Intelligent Systems)
New (User) Intent Discovery (NID) aims at discovering new intents from unlabeled data and then classifying inputs into proper (known or new) intents. In this paper, we propose a robust Pseudo label Training and source domain Joint training Network (PTJN) to generate robust pseudo labels for unlabeled data and fully utilize prior knowledge from labeled data.
We performed experiments on three public datasets: clinc, banking and stackoverflow, which have been included in our repository in the data folder ' ./data '.
Our model mainly contains three components: Extractor for feature extraction, Generator for generating pseudo labels through clustering and Corrector for refining the noisy pseudo labels generated by Generator.
- python==3.8
- pytorch==1.11.0
- transformers==4.19.2
- scipy==1.8.0
- numpy==1.21.6
- scikit-learn==1.1.1
Pre-training, training and testing our model through the bash scripts:
sh scripts/run.sh
You can also add or change parameters in run.sh (More parameters are listed in init_parameter.py)
It should be noted that the experimental results may be different because of the randomness of clustering when testing even though we fixed the random seeds.Some code references the following repositories:
If our paper or code is helpful to you, please consider citing our paper:
@article{an2023new,
title={New User Intent Discovery with Robust Pseudo Label Training and Source Domain Joint-training},
author={An, Wenbin and Tian, Feng and Chen, Ping and Zheng, Qinghua and Ding, Wei},
journal={IEEE Intelligent Systems},
year={2023},
publisher={IEEE}
}