Using the Yelp data set (https://www.yelp.com/dataset_challenge/dataset) developed a python script to complete next tasks.
- Randomly sample a small percentage (~= 1%) of users and write to disk their data in a CSV format, with headers (column names at top of file)
- Select all reviews of this sample of users and write to disk the reviews data (also in CSV format, with headers)
- Write to disk a list of user IDs (from your sample) that did not write any reviews in the last 1 year
Assumptions:
- A sample of just the user IDs can be assumed to fit in memory, but other data cannot be assumed to fit in memory.
./buld.sh
./run.sh
./build_and_run.sh