ramybaly / news-media-reliability Goto Github PK

Hello,

Thanks for releasing this repository. Can you please release the code to collect the features such as article content, YouTube text, twitter followers, etc?

Thanks!

Hello,

When running python3 classification.py --task [0] --features [1] I get the following error:

task:[0]
features:[1]
Traceback (most recent call last):
  File "classification.py", line 145, in <module>
    results = Classification(corpus, features, task)
  File "classification.py", line 72, in Classification
    X, y = GetFeaturesAndLabels(corpus, features, task)
  File "classification.py", line 67, in GetFeaturesAndLabels
    y = pd.DataFrame(np.asarray(y).reshape(-1, 1))
UnboundLocalError: local variable 'y' referenced before assignment

Why is this happening?

Any help would be much appreciated.

Thanks

Could you please add raw data i, e, news articles and wikipedia articles?

Hello,

I am having trouble reproducing the results you have in your paper. For example, for factuality detection with YouTube, twitter, and article features, your paper claims 71% accuracy using the classifier, but I get 58%.

What I run:

python3 train.py -tk fact -f articles_body_bert,articles_title_bert,twitter_profile,twitter_followers,youtube_numerical  -ds acl2020

Results I get:

10-05 23:12:36 _main_     INFO     Macro-F1: 49.796351167843326
10-05 23:12:36 _main_     INFO     Accuracy: 58.09080325960419
10-05 23:12:36 _main_     INFO     Flip Error-rate: 8.265424912689173
10-05 23:12:36 _main_     INFO     MAE: 0.5017462165308498

After inspecting further, it appears that maybe the BERT feature file is not accurate? Many of the feature values are 0 (with a very small select few negative integers). Maybe this file was corrupted somehow?