xinyadu / doc_event_role Goto Github PK
View Code? Open in Web Editor NEWLicense: MIT License
doc_event_role's People
Contributors
Stargazers
Watchers
Forkers
gillesj wangbq18 shiyanlou-015555 eric9612 fantasyoo666 mylv1222 aimaze zhichao-stone manila-devaraj leleleon12 junkroy dong4325doc_event_role's Issues
有关k-sentence reader的问题
原文还没有看完，刚看到数据构造这里。
k-sentence reader是对于一个有N条句子的篇章，构造{seq[1], ..., seq[N], seq[1] + seq[2], ..., seq[N -1] + seq[N, ...}这样的训练样本吗？这样的话样本数量会非常大，是n^2级别的，解码的时候不会特别慢吗？
What is in glove.6b.100d.txt file？
Reproducing score
Hi @xinyadu , thanks a lot for sharing the code. I am trying to reproduce the scores from Table 1 of the paper. I am grateful for your documentation and followed the steps outlined in this repo as closely as possible and am able to run the experiment all the way through. But the score I get at the end of the experiment does not match the scores reported in Table 1.
The steps I took were:
1. Environment setup/download spacy model/download
$ git clone https://github.com/xinyadu/doc_event_role.git
$ cd doc_event_role
$ touch environment.yml # add the requirements in this file
$ cat environment.yml
name: muc_seq_acl
channels:
- defaults
dependencies:
- python==3.5.6
- pip==20.1.1
- spacy==2.0.12
- cudatoolkit==9.2
- pip:
- torch==0.4.1
- pytorch-pretrained-bert==0.6.2
- typing==3.7.4.3
$ conda env create -f environment.yml
$ conda activate muc_seq_acl2. Download spacy
$ python3 -m spacy download en_core_web_sm3. Download Glove
Download glove.6B.100d.txt in doc_event_role/model/code/utils/glove.6B.100d.txt
4. Train
$ cd model/code
$ mkdir data_seq_tag_pairs
$ python gen_seq_tag_pairs.py --div train
$ mkdir model_save
$ mkdir model_out
$ python main.py --config config/example.config
$ python seq_to_extracts.py --seqfile model_out/multi_bert.out # Generate preds.jsonThe Dev scores I see during the training are:
Dev: time: 193.15s, speed: 15.20st/s; p: 28.1324, r: 70.0204, f: 40.1383
Dev: time: 124.92s, speed: 15.35st/s; p: 62.9229, r: 39.4083, f: 48.4639
Dev: time: 124.47s, speed: 15.36st/s; p: 66.1268, r: 43.9025, f: 52.7702
Dev: time: 133.18s, speed: 15.39st/s; p: 52.8802, r: 66.1970, f: 58.7939
Dev: time: 126.86s, speed: 15.37st/s; p: 63.4102, r: 48.3327, f: 54.8542
Dev: time: 124.93s, speed: 15.33st/s; p: 65.0108, r: 44.1431, f: 52.5823
Dev: time: 123.39s, speed: 15.36st/s; p: 74.8237, r: 37.4043, f: 49.8758
Dev: time: 125.21s, speed: 15.34st/s; p: 72.0788, r: 46.6892, f: 56.6702
Dev: time: 131.71s, speed: 15.31st/s; p: 57.4742, r: 69.6806, f: 62.9915
Dev: time: 129.72s, speed: 15.36st/s; p: 60.7873, r: 62.0204, f: 61.3977
Dev: time: 136.96s, speed: 15.33st/s; p: 47.4962, r: 73.2321, f: 57.6211
Dev: time: 128.23s, speed: 15.33st/s; p: 63.9869, r: 58.7949, f: 61.2811
Dev: time: 131.07s, speed: 15.36st/s; p: 58.6679, r: 62.1833, f: 60.3745
Dev: time: 126.07s, speed: 15.35st/s; p: 65.7156, r: 50.6334, f: 57.1970
Dev: time: 123.92s, speed: 15.33st/s; p: 72.3295, r: 41.9134, f: 53.0724
5. Test scores
$ python eval.py --goldfile ./data/processed/test.json --predfile ./model/code/pred.json
================Exact Match=================
Prec, Recall, F-1
PerpInd
55.1020 39.8649 46.2611
PerpOrg
40.9091 57.1429 47.6821
Target
45.3744 63.4483 52.9105
Victim
46.5753 71.5789 56.4315
Weapon
44.8980 72.1311 55.3459
MACRO average:
46.5718 60.8332 52.7557
===============Head Noun Match===============
Prec, Recall, F-1
PerpInd
59.5745 42.5676 49.6552
PerpOrg
47.7612 65.4762 55.2330
Target
60.2804 78.6207 68.2397
Victim
49.6503 72.6316 58.9815
Weapon
48.9362 75.4098 59.3548
MACRO average:
53.2405 66.9412 59.309What I am not sure about is, does this score correspond to the Multi-Granularity Reader score from Table 1 of the paper or does this correspond to some other row? Was the config file used to run the Multi-Granularity Reader experiment different from example.config?
why
None
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.