dbjpanda / tripadvisor-scraper

TripAdvisor scraper

TripAdvisor Scraper

Only for educational purposes. Use at own risk, it might violate TripAdvisor policies.

Dependencies

Install:

• Scrapy

Tested configuration

• Python 2.7.6 + Scrapy 0.24.4
• Python 2.7.6 + Scrapy 0.20.2

Usage - Have fun!

cd tripadvisor-scraper/

Scrape and save data in JSON lines format:

scrapy crawl tripadvisor-restaurant -o output/result.json

For JSON format use:

scrapy crawl tripadvisor-restaurant -o output/result.json -t json

but pay attention it doesn’t scale well for large amounts of data since incremental (aka. stream-mode)