Fetch, read, and process ground motion amplitude data.
shakemap-amp-tools is a project designed to provide a number of functions ancillary to the ShakeMap project, mostly those having to do with finding and parsing strong ground motion data and converting it into the XML data format used by ShakeMap. This repository includes the following tools:
amps2xml
Convert peak ground motions in Excel format to ShakeMap XML data structure.fdsnfetch
Retrieve data from an fdsn client (i.e. IRIS and ORFEUS).ftpfetch
Retrieve directories or files containing data from FTP url.sm2xml
Convert a directory of strong motion data files into ShakeMap xml.
Conda install:
conda install amptools
Automated install:
git clone https://github.com/usgs/shakemap-amp-tools.git
cd shakemap-amp-tools
bash intall.sh
conda activate amptools
Manual install:
conda install numpy
conda install pandas
conda install openpyxl
conda install lxml
pip install git+https://github.com/usgs/shakemap-amp-tools.git
pip install numpy
pip install pandas
pip install openpyxl
pip install lxml
pip install git+https://github.com/usgs/shakemap-amp-tools.git
Updating conda install:
conda update amptools
Updating automated install:
cd shakemap-amp-tools
git pull --ff-only https://github.com/usgs/shakemap-amp-tools.git master
bash install.sh
Updating manually installed:
pip install --upgrade git+https://github.com/usgs/shakemap-amp-tools.git
This tool converts a directory of ground motion time series data into an xml file formatted for ShakeMap. The input file must be in a format supported by amptools:
-
Cosmos
- Volume 1 and 2 are supported. No preprocessing applied.
-
CWB
- Provided by Taiwan Central Weather Bureau. No preprocessing applied.
-
DMG
- Synonymous with CSMIP. No preprocessing applied.
-
Geonet
- Provided by New Zealand GNS. Currently Volume 1 and 2 are supported.
- Data is converted from mm/s2 to cm/s2 when read in.
-
KNET and KIKNET
- Provided by Japan National Research Institute for Earth Science and Disaster Resilience.
- Time is converted from Japanese local time to UTC time (9 hour time shift).
- A 15 second time delay (added by logger) is removed.
- The data is detrended (linear and demean) as specified by documentation.
-
Obspy
- All obspy supported formats are acceptable when accompanied by a JSON file containing metadata.
-
SMC
- Standard U.S.G.S. format specified by the National Strong Motion Project. No preprocessing applied.
-
USC
- Provided by Los Angeles Basin Seismic Network, University of Southern California. No preprocessing applied.
- Volume 1 is supported. No preprocessing applied.
This tool allows a user to fetch raw waveforms from an fdsn client. Clipped waveforms are automatically removed. Each waveform is also demeaned and the instrument response is removed. Data is output as miniseed or sac, along with an accompanying metadata file for each station.
This tool allows a user to fetch a file or directory of files from an FTP server. If a username and password are required (KNET, for example) they can be provided as command line options.
This tool converts an excel file of peak ground motion data into an xml file formatted for use by ShakeMap. Excel spreadsheets must contain only tabular (rows/columns) data.
A "complete" peak ground motion spreadsheet would look like this:
reference | Mexican National Seismic Service report dated 2010-01-01 | ||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
station | NAME | NETID | SOURCE | LAT | LON | ELEV | DISTANCE | INTENSITY | H1 | H2 | Z | ||||||||||||
PGA | PGV | SA(0.3) | SA(1.0) | SA(3.0) | PGA | PGV | SA(0.3) | SA(1.0) | SA(3.0) | PGA | PGV | SA(0.3) | SA(1.0) | SA(3.0) | |||||||||
ACAM | Guanajuato | MX | National Seismic Service | 20.043186 | -100.716777 | 5.1 | 913.1 | 0.72 | 4.48 | 1.31 | 5.12 | 7.74 | 2.93 | 4.928 | 1.44 | 5.632 | 8.514 | 3.223 | 4.256 | 1.24 | 4.864 | 7.353 | 2.784 |
A minimum intensity only peak ground motion table would look like this:
reference | Jones and Smith, 2007. Journal of Seismology. | |||
---|---|---|---|---|
station | lat | lon | network | intensity |
ABC | 20.123 | -118.123 | MX | 4.5 |
A minimum pga only peak ground motion table would look like this:
reference | Jones and Smith, 2007. Journal of Seismology. | |||
---|---|---|---|---|
station | lat | lon | network | unk |
pga | ||||
ABC | 20.123 | -118.123 | MX | 0.5 |
The reference section in the first row is mandatory, and should consist of a cell in the first column with "reference" in it, and the cell in the second column containing the reference string.
The required columns are:
- station: Usually a short abbreviated code.
- lat: Latitude of the station where data was collected.
- lon: Longitude of the station where data was collected.
- network: Usually a short code indicating the seismic network who collected the data.
In addition to these required columns, there must be additionally at least:
- intensity: Modified Mercalli Intensity (MMI) at the station location.
AND/OR:
- channel (more on this below), containing at least one of the following:
- pga: Peak Ground Acceleration, in units of %g.
- pgv: Peak Ground Velocity, in units of cm/s.
- psa03: Peak Spectral Acceleration at 0.3 seconds, in units of %g.
- psa10: Peak Spectral Acceleration at 1.0 seconds, in units of %g.
- psa30: Peak Spectral Acceleration at 3.0 seconds, in units of %g.
The channel columns can be one of three different naming schemes:
- XXE, XXN, XXZ: Three channels named according to the scheme found at http://www.fdsn.org/seed_manual/SEEDManual_V2.4_Appendix-A.pdf where XXE/XXN are horizontal channels, and XXZ is the vertical channels.
- H1, H2, Z: Three channels where the direction of the channel is not known, where H1/H2 are horizontal channels and Z is the vertical channel.
- UNK: The "channel" to use when only summary peak ground motion values are known.
For the "XXE/XXN/XXZ" or "H1/H2/Z" naming schemes, at least one of the horizontal channels must be included.
Links to example spreadsheets:
The data file readers are modeled after obspy file readers, and have a standard interface.
Data file readers are located in amptools/io/[format]/core.py
.
This core.py module should take the following form:
def is_format(filename): # code to examine candidate file and determine if it is of the type specified. # return True if file is correct type, False otherwise. def read_format(filename,**kwargs): # code to read file and return an obspy Stream object.
In order to add an intensity measurement types (IMT) calculation, add a file with the type as the file name under pgm/imt/. The calculation should be called calculate_[TYPE], where [TYPE] matches the file name. The argument should always be stream. The second function should always be imcs (a list of requested components). All other functions in the file should be hidden by an underscore (example def _get_horizontals(stream)
). The calculate function should return a dictionary of each component and the resulting values. Example:
{ 'HN2': 81.234, 'GMROTD0.0': 86.784, 'GMROTD100.0': 96.446, 'HN1': 99.249, 'HNZ': 183.772, 'GMROTD50.0': 92.177, 'GREATER_OF_TWO_HORIZONTALS': 99.249 }
StationSummary should be updated to handle the new IMT in gather_pgms
.
In order to add an intensity measurement component (IMC) calculation, add a file with the component as the file name under pgm/imc/. The calculation should be called calculate_[COMPONENT], where [COMPONENT] matches the file name. The argument should always be stream. All other functions in the file should be hidden by an underscore (example def _get_horizontals(stream)
). The calculate function should return a single value or a dictionary of each component and the resulting values. Imt calculations should be updated to handle a dictionary if one is returned. Otherwise, single values will automatically be handled.
Greater of two horizontals example: 99.249
Channels example:
{ 'HN1': 99.249, 'HN2': 81.234, 'HNZ': 183.772 }
GMRotD example:
{ 0.0: 103.299, 50.0: 119.925, 100.0: 125.406 }