First Street Foundation API Access Documentation (Python 3.7+)

The First Street Foundation API Access (Python) is a wrapper used to bulk extract flood data from the First Street Foundation API

Notice: This API wrapper is subject to change.

Please see the wiki pages here for the most up to date information.

Installation
- Running the Project - Method 1: Through the Command Line
  - Command Line Argument Details
- Running the Project - Method 2: Through the Client
Products
- Location
- Probability
- Historic
- Adaptation
- Fema
- Environmental
- Tile
Examples
CSV File Output:
- CSV File Name:
- CSV File Content
Updating the Project to the Newest Version:
License

Installation

NOTE: This project requires Python 3.7+ to run.

Go to the Python page (https://www.python.org/downloads/) and download then install Python version 3. Make sure that the checkbox is checked for Python to be added to the PATH

A more detailed and in-depth guide on how to install Python can be found here
[Optional] Open a new powershell console / bash terminaland create and activate a new virtual environment in the project directory:
```
python -m venv /path/to/new/virtual/environment

cd /path/to/project

.\venv\Scripts\activate
```

Use pip to install the project

cd /path/to/project

pip install fsf-api-access-python

The project is now setup and can be ran through one of the two methods below. See Products for additional details on how to extract flood data products from the API.
```
Example File Structure:

.
├── venv
├── my_script.py
```

Running the Project - Method 1: Through the Command Line

[Reminder] Keep your API key safe, and do not share it with others!

[NOTE] This method will always generate a CSV.

MacOS or Linux

[Required] Open a bash terminal and set an Environmental Variable with the variable_name as FSF_API_KEY and the variable_value with the API_KEY with the command
```
export FSF_API_KEY=your personal API key
```
Windows

[Required] Set an Environmental Variable with the variable_name as FSF_API_KEY and the variable_value with the API_KEY.
Open/Re-open a powershell console / bash terminal and navigate to the project directory. Next, call one of the methods described below in the Products section through the command line. See the Examples section for more examples.
```
cd /path/to/project

python -m firststreet -p <product>.<product_subtype> -i <search_item> -f <file_name> -l <lookup_type>
```

Command Line Argument Details:

-p/--product PRODUCT: [REQUIRED] The product to call from the API

Example: -p location.get_detail
[-api_key/--api_key API_KEY]: [OPTIONAL] If step 1 is skipped or unavailable, this argument can take the API_KEY instead.

Example: -a XXXXXXXXXXXXXXXXXXX
[-v/--version VERSION]: [OPTIONAL] The version of the API to call. Defaults to the current version.

Example: -v v1
[-i/--search_items SEARCH_ITEM]: [OPTIONAL] The Search Items to search for with the product.

NOTE THE LIST MUST BE A SEPRATED WITH A SEMICOLON (;) INSTEAD OF A COMMA

Example: -i 541114211;541229206
[-l/--location LOOKUP_TYPE]: [OPTIONAL] The lookup location type (property, neighborhood, city, zcta, tract, county, cd, state)

Example: -l property
[-connection_limit/--connection_limit CONNECTION_LIMIT]: [OPTIONAL] The max number of concurrent connections to make to the API at the same time. This is does not affect the number of FSIDs that can be pulled. Defaults to 100 connections

Example: -connection_limit 20000
[-rate_limit/--rate_limit RATE_LIMIT]: [OPTIONAL] The max number of requests during the rate limit period. Defaults to 20000 requests

Example: -rate_limit 1
[-rate_period/--rate_period RATE_PERIOD]: [OPTIONAL] The period of time for the rate limit. Defaults to 1 second

Example: -rate_period 20
[-log/--log LOG]: [OPTIONAL] To log info output to the console or not. Defaults to True

Example: -l False
[-e/--extra_param EXTRA_PARAM]: [OPTIONAL] Adds the argument to the end of the endpoint call

Example: -e extra_param
[-year/--year YEAR]: [OPTIONAL] The year to use for the Probability Depth Tile product

Example: -year 2050
[-return_period/--return_period RETURN_PERIOD]: [OPTIONAL] The return period to use for the Probability Depth Tile product

Example: -return_period 500
[-event_id/--event_id EVENT_ID]: [OPTIONAL] The event id to use for the Historic Event Tile product

Example: -event_id 3
[-f/--file FILE]: [OPTIONAL] A file of Search Items (one per line) to search for with the product

Example: -f sample.txt

Content of a sample text file. Note that the file must be in the same directory as the project:
```
541114211
540456284
541229206
540563324
541262690
540651172
```

Running the Project - Method 2: Through the Client

[Reminder] Keep your API key safe, and do not share it with others!

[NOTE] This method will NOT generate a CSV by default. An argument must be passed to generate a CSV.

Create a new Python script (by using notepad or any other text editor) and initialize a First Street Foundation API Client.
```
# Contents of my_script.py
import firststreet
fs = firststreet.FirstStreet("api-key")
```
Client Initialization Details
```
firststreet.FirstStreet(api_key, [connection_limit], [rate_limit], [rate_period], [version], [log])
```
Arguments
- api_key string: The assigned API key to access the API.
- connection_limit int=100: The max number of connections to make
- rate_limit int=20000: The max number of requests during the period
- rate_period int=1: The period of time for the limit
- version string= v1: The version of the API to access. Defaults to the current version.
- log bool= True: Setting for whether to log info or not. Defaults to True.
Call one of the methods described below in the Products section with the required arguments. See the Examples section for more examples.
```
fs.<product>.<product_subtype>(<search_items: list>, <lookup_type: string>, <csv: boolean>)
```
OR
Run the python script.

If you are using Jupyter Notebook

Add the following to the top of the file before the client initialization to allow the jupyter notebook and download loops to work correctly
```
# Setup For Notebook only
import nest_asyncio
nest_asyncio.apply()
```

Products

More information on each product can be found at the First Street Foundation API Data Dictionary

Search Item:

For every product, a list or file of search items must be provided to the product call. There are 3 types of search items corresponding to the 3 types of lookups. (More information on the Lookup types can be found on the Lookups Page on the First Street Foundation API Data Dictionary )

FSID (int): The simplest type of lookup is a fsid lookup. If you know the fsid of the specific location, you can navigate directly to the specific product's information using the location's fsid. Example FSID: 18
Lat/Lng (tuple of int): Provide a tuple of latitude, longitude pairs will check to see if that point intersects within a boundary of a parcel from the database. Example lat/lng: (40.7079652311, -74.0021455387)
Address (string): Pass the address in your request to retrieve a reverse geocode location lookup. LocationType is required for address lookup. An address can be a city name, home address, or business address. City and State need to be included within the address query. Example Address: New York, NY

Example list of search items:

lst = [362493883, (40.792505, -73.951949), "1220 5th Ave, New York, NY"]

Example file of search items:

362493883
(40.792505, -73.951949)
1220 5th Ave, New York, NY

Location

The Location API provides Detail and Summary data for the given SearchItems. (More information on the Location product can be found on the Location Page on the First Street Foundation API Data Dictionary )

location.<method>

get_detail(search_items list/file, location_type string, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Location Detail product for the given IDs, location_type, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_summary(search_items list/file, location_type string, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Location Summary product for the given IDs, location_type, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call

Probability

The Probability API provides Depth, Chance, Cumulative, Count data for the given SearchItems. (More information on the Probability product can be found on the Probability Page on the First Street Foundation API Data Dictionary )

probability.<method>

get_depth(search_items list/file, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Probability Depth product for the given IDs, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_chance(search_items list/file, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Probability Chance product for the given IDs, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_count_summary(search_items list/file, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Probability Count-Summary product for the given IDs, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_cumulative(search_items list/file, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Probability Cumulative product for the given IDs, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_count(search_items list/file, location_type string, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Probability Count product for the given IDs, location_type, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call

Historic

The Historic API provides Summary and Event data for the given SearchItems. (More information on the Historic product can be found on the Historic Page on the First Street Foundation API Data Dictionary )

historic.<method>

get_event(search_items list/file, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Historic Event product for the given historic IDs, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_events_by_location(search_items list/file, location_type string, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Historic Detail product for the given IDs, location_type, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_summary(search_items list/file, location_type string, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Historic Summary product for the given IDs, location_type, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call

Adaptation

(More information on the Adaptation product can be found on the Adaptation Page on the First Street Foundation API Data Dictionary )

The Adaptation API provides Summary and Project data for the given SearchItems.

adaptation.<method>

get_project(search_items list/file, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Adaptation Project product for the given adaptation IDs, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_details_by_location(search_items list/file, location_type string, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Adaptation Project product for the given IDs, location_type, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call
get_summary(search_items list/file, location_type string, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Adaptation Summary product for the given IDs, location_type, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call

Fema

The Fema API provides NFIP data for the given SearchItems. (More information on the Fema NFIP product can be found on the Fema NFIP Page on the First Street Foundation API Data Dictionary )

fema.<method>

get_nfip(search_items list/file, location_type string, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Fema NFIP product for the given IDs, location_type, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call

Environmental

The Environmental API provides Precipitation data for the given SearchItems. (More information on the Environmental Precipitation product can be found on the Environmental Precipitation Page on the First Street Foundation API Data Dictionary )

environmental.<method>

get_precipitation(search_items list/file, csv bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Environmental Precipitation product for the given county IDs, and optionally creates a csv file. Arguments provided to extra_param will be appened to the end of the endpoint call

Tiles

The Flood Tiles product give the ability to customize maps by layering on flood tiles. (More information on the Tile product can be found on the Tile Page on the First Street Foundation API Data Dictionary )

tile.<method>

get_probability_depth(coordinate list of tuple of int [(z, x, y)], year int, return_period int, image bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Probability Depth Tile product for the given coordinates, and optionally creates an image file
get_historic_event(coordinate list of tuple of int [(z, x, y)], event_id int, image bool, [output_dir str='cwd'], [extra_param str=None]) - Returns an array of Historic Event Tile product for the given coordinates, and optionally creates an image file

Examples

[Reminder] Keep your API key safe, and do not share it with others!

Single FSID Extraction Through the Client:

# Contents of sample.py
import firststreet
fs = firststreet.FirstStreet("api-key")

fsids = [39035103400]
probability = fs.probability.get_count(fsids, "tract")

print(probability[0].fsid)
print(probability[0].count)

Multiple FSIDs Extraction Through the Client:

# Contents of sample.py
import firststreet
fs = firststreet.FirstStreet("api-key")

fsids = [1912000, 1979140]
details = fs.location.get_detail(fsids, "city")

print(details[0].fsid)
print(details[0].county)
print(details[1].fsid)

Adaptation detail Extraction to CSV Through the Client:

# Contents of sample.py
import firststreet
fs = firststreet.FirstStreet("api-key")

adaptation = fs.adaptation.get_detail([29], csv=True)

print(adaptation[0].adaptationId)
print(adaptation[0].name)

Output File:

 adaptationId,name,type,scenario,conveyance,returnPeriod
 29,Riverfront Park,bioswale,fluvial,False,500
 29,Riverfront Park,bioswale,pluvial,False,500
 29,Riverfront Park,detention basin,fluvial,False,500
 29,Riverfront Park,detention basin,pluvial,False,500
 29,Riverfront Park,levee,fluvial,False,500
 29,Riverfront Park,levee,pluvial,False,500
 29,Riverfront Park,pervious pavement,fluvial,False,500
 29,Riverfront Park,pervious pavement,pluvial,False,500

Multiple FSIDs Extraction Using a File Through the Client:

Content of sample.txt:

# Contents of sample.py
import firststreet
fs = firststreet.FirstStreet("api-key")

details = fs.location.get_detail("sample.txt", "property")

print(details[0].fsid)
print(details[0].county)
print(details[1].fsid)

Different Location Types Through the Client:

# Contents of sample.py
import firststreet
fs = firststreet.FirstStreet("api-key")

lst = [362493883, (40.792505, -73.951949), "1220 5th Ave, New York, NY"]
location = fs.location.get_detail(lst, location_type="property", csv=True)

print(location[0].fsid)
print(location[0].route)
print(location[1].fsid)
print(location[1].route)
print(location[2].fsid)
print(location[2].route)

Single FSID Extraction to CSV Through the Command Line:

python -m firststreet -p probability.get_depth -i 390000227

Multiple FSIDs Extraction to CSV Through the Command Line:

python -m firststreet -p historic.get_summary -i 1912000;1979140 -l property

Bulk FSIDs Extraction From File to CSV Through the Command Line:

Content of sample.txt:

python -m firststreet -p location.get_summary -f sample.txt -l property

Output File 2020_06_10_20_33_12_location_summary_property.csv:

fsid,floodFactor,riskDirection,environmentalRisk,historic,adaptation
541114211,1,0,1,0,0
540456284,9,0,1,0,0
541229206,9,1,1,0,0
540563324,1,0,1,0,0
541262690,1,0,1,0,0
540651172,1,0,1,0,0

Different Location Types Through the Command Line:

Note that the separator for each item must be a semicolon ;

python -m firststreet -p location.get_detail -i "362493883;(40.792505,-73.951949);1220 5th Ave, New York, NY" -l property

Different Locatio Types From File to CSV Through Command Line:

Content of sample.txt:

362493883
(40.792505, -73.951949)
1220 5th Ave, New York, NY

python -m firststreet -p location.get_summary -f sample.txt -l property

Getting a Tile from the API Tile Product Through the Client:

# Contents of sample.py
import firststreet
fs = firststreet.FirstStreet("api-key")

fs.tile.get_probability_depth(year=2050, return_period=500, coordinate=[(14, 2633, 5694)], image=True)
tile_image = fs.tile.get_historic_event(event_id=2, coordinate=[(12, 942, 1715)], image=False)

Getting a Tile from the API Tile Product Through the Command Line:

Note that the separator for each coordinate must be a semicolon ;, and the spacing within the coordinates

python -m firststreet -p tile.get_probability_depth -i (12,942,1715);(11,942,1715) -year 2050 -return_period 500
python -m firststreet -p tile.get_historic_event -i (12,942,1715) -event_id 14

CSV File Output:

Any product above can be additionally exported to a CSV file for further usage if the csv boolean is set during the product call, or any call using the command line. The extracted can be found in the output_data directory of the project folder (if at least one CSV has been extracted).

CSV File Name:

The file name will be in the format of YYYY_MM_DD_HH_mm_SS_product_subtype_lookup.csv.

Ex:

2020_06_10_17_29_49_adaptation_detail.csv
2020_06_10_17_33_56_adaptation_summary_property.csv

CSV File Content

The contents of the CSV file will follow similar formats as the First Street Foundation API - V1.0 Overview and Dictionary, but all lists will be expanded to a flat file. For any values that are null or not available, <NA> will be used.

If any of the input FSIDs are invalid, a valid_id column will appear with the invalid FSIDs marked with False

Ex:

fsid,year,returnPeriod,bin,low,mid,high
7935,2020,20,20,<NA>,2,<NA>
7935,2020,20,50,<NA>,1,<NA>
7935,2020,20,55,<NA>,2,<NA>
7935,2020,20,65,<NA>,2,<NA>
7935,2020,20,75,<NA>,2,<NA>
7935,2020,20,95,<NA>,1,<NA>
7935,2020,20,100,<NA>,1,<NA>
...

Updating the Project to the Newest Version:

If an update is made to this project, you will need to upgrade through PyPi

Open a new powershell console / bash terminal, navigate to the project, and run the upgrade command:
```
cd /path/to/project

.\venv\Scripts\activate

pip install --upgrade fsf_api_access_python
```
The project should now be updated to the newest version

License

MIT License

Copyright (c) 2020 First Street Foundation

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.

fagan2888 / fsf_api_access_python Goto Github PK

fsf_api_access_python's Introduction

First Street Foundation API Access Documentation (Python 3.7+)

Table of contents

MacOS or Linux

Windows

fsf_api_access_python's People

Contributors

Recommend Projects

Recommend Topics

Recommend Org

Jobs