KeyError: 'pdf_json_files' about paperetl

[davidmezzetti]

Based on the date of 2020-03-27, are you trying to run on CORD-19 data from that date? The format changed on 2020-05-12 and this method only supports data dumps from that date and on:

https://ai2-semanticscholar-cord-19.s3-us-west-2.amazonaws.com/historical_releases.html

Unless you have a specific reason, I would use the latest dump from the link above.

[SeekPoint]

ok, I just want try it on a small file

[davidmezzetti]

Best thing to do would be to download the latest dump, extract it and filter a few rows from metadata.csv

For example

tar -xvzf cord-19_2020-08-12.tar.gz
cd 2020-08-12
mv metadata.csv metadata.csv.bkup
head -500 metadata.csv.bkup > metadata.csv

[SeekPoint]

it works