several27 / fakenewscorpus Goto Github PK
View Code? Open in Web Editor NEWA dataset of millions of news articles scraped from a curated list of data sources.
License: Apache License 2.0
A dataset of millions of news articles scraped from a curated list of data sources.
License: Apache License 2.0
Hello!
Thank you for this huge dataset!
Currently working with it ("fake" and "reliable" labels only for now), I will probably find some problems in it, the majors of which I will post here :).
First one :
contents of openmagazines.com articles are mainly the following:
"This website uses cookies to improve your experience. We'll assume you're ok with this, but you can opt-out if you wish. Accept"
for 1023 entries out of 1081.
As a fast fix, automatically replacing this content with None or "" would be cleaner :).
Hi,
I am trying to save the data into Cassandra database but it cannot interpret the CSV. After that I tried to check it with Excel and the pandas dataframe but both said that the file does not have a valid CSV format.
Could you help me to find the way how to store the data to Cassandra?
Hi @several27!
I'm doing some research on fake news datasets for automatic detection and your corpus is the most complete I've found, it can be really useful for my study! But I don't know if you have a desirable way to cite your work of if it's published somewhere?
Do you have an email that I can contact you?
Thanks!
Used this command to create a combined zip from the split parts in a directory:
zip -F (name of last part of archive, which will end with .zip, not .z0X) --out (output name of compiled archive).zip
and unzip (archive name).zip to unpack it.Unpacking gives an error after an extraction of 2.8 GB of data.
Also used p7zip to unpack individual files or the combined file and get the error
@several27 It'd be great if you could help me with this.
Hi,
Would it be possible for you to host the dataset somewhere else as a more accessible download?
I have tried downloading the dataset via awscli, but it throws an error that indicates a permission and/or region mismatch. Is the bucket still public?
Awesome work, but in the files we can see a type "rumor" that is not documented in the readme...
Hi @several27 ,
I am currently using your corpus for an NLP project and was wondering if you had the dates for when each article was published available. Otherwise I was wondering if you had the raw HTML for each article that I can download; I can retrieve these dates from the raw HTML. This is because a lot of the domains are dead and I can't look on the internet for these dates anymore.
Thanks,
Changxiao
Dear Maciej,
Thanks a lot for making this amazing dataset available! :D I have one quick question and a comment.
Thanks,
Jisun
How to read the file? It's encoded somehow probably?
Is there any field describing when an article was written?
Hello,
How do you label the article? For example, the data collected from the URL http://beforeitsnews.com/awakening-start-here/2018/01/awakening-of-12-strands-of-dna-reconnecting-with-you-movie-10623.html there is no information that the article is fake.
Please explain if I miss something.
Hello, after I extracted the news.csv.zip file and I opened it in Excel, it only showed me a grayed out screen. Excel did not recognize a file being open since I could not Save As or do any other actions. I have tried opening the CSV using another method from the Data tab then get external data, which did not work as well. I believe that the files you have put maybe corrupted. I would appreciate it if you could update it. Or if anyone knows a solution to my problem do respond.
Thank You
Hi,
When I am trying to download the file using wget command on google colab and I am getting below error:-
--2020-01-03 04:14:40-- https://storage.googleapis.com/researchably-fake-news-recognition/news_cleaned_2018_02_13.csv.zip
Resolving storage.googleapis.com (storage.googleapis.com)... 108.177.97.128, 2404:6800:4008:c03::80
Connecting to storage.googleapis.com (storage.googleapis.com)|108.177.97.128|:443... connected.
HTTP request sent, awaiting response... 403 Forbidden
2020-01-03 04:14:40 ERROR 403: Forbidden.
Hello can someone can give me some tips on how to split the file into smaller files?
Thanks.
Is the data still open for public? I click the link but can't download the data.
Any chance of getting the head of the dataset in the readme. The file is so big I've been wasting a lot of time just to access and organize it. Having the first 5-10 rows would have saved me a headache. I'm sure someone else could benefit from this.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.