antct / cil

Pytorch implementation of "CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction"

CIL

Code base for "CIL: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction".

Env

For distributed parallel training, we use the library huggingface/accelerate for convenience, and its configuration is in the file ddp.yaml (num_processes: number of available GPUs ~ CUDA_VISIBLE_DEVICES).

pip install -r requirements.txt	

Data

We follow the data format in OpenNRE, each dataset folder contains the following files:

nyt10d_rel2id.json: Mapping of relation category to relation ID

{
  "NA": 0,
  "/location/neighborhood/neighborhood_of": 1,
  ...
  "/film/film_festival/location": 52
}

nyt10d_(train|dev|test).txt: Each line of data is in json format ↓

{
  "text": "sen. charles e. schumer called on federal safety officials yesterday to reopen their investigation into the fatal crash of a passenger jet in belle harbor , queens , because equipment failure , not pilot error , might have been the cause .",
  "relation": "/location/location/contains",
  "h": {
    "id": "m.0ccvx",
    "name": "queens",
    "pos": [157, 163]
  },
  "t": {
    "id": "m.05gf08",
    "name": "belle harbor",
    "pos": [142, 154]
  }
}

Run

1. Download dataset (refer to benchmark/nyt10d/readme.md) and pretrained bert-base-uncased:

bash download_bert.sh

2. Run script aug.py to generate positive pair samples for training dataset (Line 18: dataset name):

python aug.py

3. Run script in folder scripts to train and evaluate the DSRE models:

bash train_nyt10d.sh SEED

4. Run script plot.py in folder pr to draw pr curves of the trained model.

python plot.py

PR Curve

The PR curves (NYT10-D) of all baseline models are in Google Drive.

Cite

@inproceedings{chen-etal-2021-cil,
    title = "{CIL}: Contrastive Instance Learning Framework for Distantly Supervised Relation Extraction",
    author = "Chen, Tao  and
      Shi, Haizhou  and
      Tang, Siliang  and
      Chen, Zhigang  and
      Wu, Fei  and
      Zhuang, Yueting",
    booktitle = "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)",
    month = aug,
    year = "2021",
    address = "Online",
    publisher = "Association for Computational Linguistics",
    url = "https://aclanthology.org/2021.acl-long.483",
    doi = "10.18653/v1/2021.acl-long.483",
    pages = "6191--6200",
    abstract = "The journey of reducing noise from distant supervision (DS) generated training data has been started since the DS was first introduced into the relation extraction (RE) task. For the past decade, researchers apply the multi-instance learning (MIL) framework to find the most reliable feature from a bag of sentences. Although the pattern of MIL bags can greatly reduce DS noise, it fails to represent many other useful sentence features in the datasets. In many cases, these sentence features can only be acquired by extra sentence-level human annotation with heavy costs. Therefore, the performance of distantly supervised RE models is bounded. In this paper, we go beyond typical MIL framework and propose a novel contrastive instance learning (CIL) framework. Specifically, we regard the initial MIL as the relational triple encoder and constraint positive pairs against negative pairs for each instance. Experiments demonstrate the effectiveness of our proposed framework, with significant improvements over the previous methods on NYT10, GDS and KBP.",
}

cil's Issues

关于代码的2个小问题

你好，看了下代码，有2个小问题：

1. In bag_attention.py, line 59~60, how to calculate cosine_similarity between two tensors with different size ((bag, H, B) and (B, 1, H, B))? I try myself that it did work, but I don't know why could this happen (and also what's the detailed calculation process).
2. 在计算CIL的时候，正例是instance ~ augmented instance， 负例是instance ~ other bag representation，看上去跟传统CL的排他性略有不同（即除了正例外都是负例），能理解是DS的数据中有噪音导致无法实现，但对于效果或者模型的训练会不会有什么影响？想知道作者在这方面有什么看法。谢谢！

GIDS数据集预训练模型

请问可否提供一下GIDS数据集下CIL模型的PR曲线获取预训练模型？

torch.nn.modules.module.ModuleAttributeError: 'BagAttention' object has no attribute 'module'

需要把unprocessed data处理成processed data才能运行aug.py吗？

你好，我看到除了nyt10d外的数据都是unprocessed data，需要自己把他们处理成processed data的格式，才能运行aug.py吗？谢谢

KBP数据集预处理

您好，感谢您分享您的源码。

关于KBP数据预处理我有以下2个问题想向您请教：

1. KBP未处理的数据文件中，“brown”文件的含义和作用是什么？
2. 在readme中给出的处理好的KBP样例中，实体id样式为"1-a"、"1-b"这种（如下所示），请问您是如何定义的呢？我猜测是跟未处理的源格式有一定联系，不知是否正确。

kbp_train.txt kbp_dev.txt kbp_test.txt
{"text": "An American Civil rights activist , Martin Luther King , Jr. was assassinated in 1968 by James Earl Ray in Memphis , Tennessee .", 
"relation": "no_relation",
 "h": {"id": "1-a", "name": "American Civil", "pos": [3, 17]}, 
"t": {"id": "1-b", "name": "James Earl Ray", "pos": [89, 103]}}

感谢您的解答！