lulzzz / ecocrop-scraper

Using R to scrape the FAO EcoCrop database

EcoCrop ScrapeR

Scraping the FAO EcoCrop database

The Food and Agriculture Organization of the United Nations (FAO) produced a Crop Ecological Requirements Database (EcoCrop) (explanation here) (database here) with myriad data about over 2500 crop species.

These data have proven useuful for a variety of projects (1,2, 3..) but the website offers no download options and query options are limited certain types actions.

To gain better access to the dataset three scripts are used to output .csv files that scrape certain portions of the data for a given crop.

The EcoCrop_Scraper.R script scrapes the 'Data sheet' page (example for Okra), which contains climatic (rainfall, temperature), soil (pH, depth, etc), and other data for each crop and outputs to cropbasics_scrape.csv. To function it requires the datasheeturl.csv which is a preproduced list of all the crop urls created by appending the crop codes, found by pulling the full species list by selecting 'all' plants on the search page, to the datasheet url prefix.

The Usery.R script scrapes listed uses for the crop and outputs the listing for all crops into crop_uses.csv, and also requires the prepdocuced datasheeturl.csv to function.

The Viewcrop.R script scrapes additional data (common names, data sources, etc) from the 'View crop' page (example for Okra) for each crop and outputs to crop_view_data.csv. To function it requires the viewcropurl.csv which is a preproduced list of 'View crop' page urls for all crops created by appending the crop codes, found by pulling the full species list by selecting 'all' plants on the search page, to the viewcrop url prefix.