jpvantassel / hvsrpy

A Python package for Horizontal-to-Vertical (H/V, HVSR) Spectral Ratio Processing.

hvsrpy - A Python package for horizontal-to-vertical spectral ratio processing

Joseph P. Vantassel, jpvantassel.com

Table of Contents

• About hvsrpy
• Why use hvsrpy
• A Comparison of hvsrpy with Geopsy
• Getting Started
• Additional Comparisons between hvsrpy and Geopsy
  • Multiple Windows
  • Single Window

About hvsrpy

hvsrpy is an open-source Python package for performing horizontal-to-vertical spectral ratio (HVSR) processing of microtremor and earthquake recordings. hvsrpy was developed by Joseph P. Vantassel with contributions from Dana M. Brannon under the supervision of Professor Brady R. Cox at The University of Texas at Austin. hvsrpy continues to be developed and maintained by Joseph P. Vantassel and his research group at Virginia Tech.

If you use hvsrpy in your research or consulting, we ask you please cite the following:

Joseph Vantassel. (2020). jpvantassel/hvsrpy: latest (Concept). Zenodo. http://doi.org/10.5281/zenodo.3666956

Note: For software, version specific citations should be preferred to general concept citations, such as that listed above. To generate a version specific citation for hvsrpy, please use the citation tool on the hvsrpy archive.

hvsrpy implements the work of many others. Therefore we strongly encourage users to cite the original work alongside hvsrpy. Below we list key features and their associated citation.

Lognormal statistics and fully-automated frequency-domain window-rejection algorithm.

Cox, B. R., Cheng, T., Vantassel, J. P., & Manuel, L. (2020). "A statistical representation and frequency-domain window-rejection algorithm for single-station HVSR measurements. Geophysical Journal International, 221(3), 2170–2183. https://doi.org/10.1093/gji/ggaa119

Statistical approach to incorporate azimuthal variability.

Cheng, T., Cox, B. R., Vantassel, J. P., and Manuel, L. (2020). "A statistical approach to account for azimuthal variability in single-station HVSR measurements." Geophysical Journal International, 223(2), 1040–1053. https://doi.org/10.1093/gji/ggaa342

Statistical approach to incorporate spatial variability.

Cheng, T., Hallal, M. M., Vantassel, J. P., and Cox, B. R., (2021). "Estimating Unbiased Statistics for Fundamental Site Frequency Using Spatially Distributed HVSR Measurements and Voronoi Tessellation. J. Geotech. Geoenviron. Eng. 147, 04021068. https://doi.org/10.1061/(ASCE)GT.1943-5606.0002551

SESAME clarity and reliability criteria for microtremor HVSR.

SESAME. (2004). Guidelines for the Implementation of the H/V Spectral Ratio Technique on Ambient Vibrations Measurements, Processing, and Interpretation. European Commission - Research General Directorate, 62, European Commission - Research General Directorate.

Computation of HVSR under the diffuse field assumption.

Sánchez-Sesma, F.J., Rodríguez, M., Iturrarán-Viveros, U., Luzón, F., Campillo, M., Margerin, L., García-Jerez, A., Suarez, M., Santoyo, M.A., Rodríguez-Castellanos, A. (2011). A theory for microtremor H/V spectral ratio: application for a layered medium: Theory for microtremor H/V spectral ratio. Geophysical Journal International 186, 221–225. https://doi.org/10.1111/j.1365-246X.2011.05064.x

Power spectral density (PSD) calculation.

Welch, P., (1967). The use of fast Fourier transform for the estimation of power spectra: a method based on time averaging over short, modified periodograms. IEEE Transactions on audio and electroacoustics, 15(2), pp.70-73.

New high noise model (NHNM) and new low noise model (NLNM).

Peterson, J., (1993). Observation and Modeling of Seismic Background Noise (Open-File Report No. 93–322). U.S. Geologic Survey, Albuquerque, New Mexico.

hvsrpy would not exist without the help of many others. As a display of our gratitude, we thank them individually here.

Why use hvsrpy

hvsrpy contains features not currently available in any other commercial or open-source software, including:

• Support of a broad range of microtremor and earthquake data formats including: miniSEED, SAF, MiniShark, SAC, GCF, and PEER.
• Multiple methods to combine the horizontal components including: arithmetic_mean, squared-average, quadratic-mean, geometric-mean, total-horizontal-energy, vector-summation, maximum-horizontal-value, rotD50, or single-azimuth.
• Ability to use a lognormal distribution for the site resonant frequency (fn) so the uncertainty in fn can be represented consistently in frequency and period.
• Direct access to the raw HVSR data from each time window and azimuth (i.e., not only the mean/median curve) through Python objects and cross-platform text-based output format.
• Calculate statistics on fn that incorporates azimuthal variability.
• Ability to develop rigorous and unbiased spatial statistics to combine spatially distributed HVSR measurements.
• Multiple automated window rejection algorithms including the frequency-domain window-rejection algorithm by Cox et al. (2020).
• Automated checking of the SESAME (2004) peak reliability and clarity criteria.
• A command line interface (CLI) for parallel batch-style processing using multiprocessing.

Example output from hvsrpy when considering the geometric-mean of the horizontal components

	Lognormal Median	Lognormal Standard Deviation
Site Frequency, fn	0.67	0.09
Site Period, Tn	1.50	0.09

Example output from hvsrpy when considering azimuthal variability

	Lognormal Median	Lognormal Standard Deviation
Site Frequency, fn,az	0.67	0.07
Site Period, Tn,az	1.50	0.07

Example output from hvsrpy when considering spatial variability

	Lognormal Median	Lognormal Standard Deviation
Site Frequency, fn,xy	0.58	0.15
Site Period, Tn,xy	1.74	0.15

A comparison of hvsrpy with Geopsy

Some of the functionality available in hvsrpy overlaps with the popular open-source software geopsy. Therefore, to encourage standardization, wherever their functionality coincides we have sought to ensure consistency. Two such comparisons are shown below. One for a single time window (top) and one for multiple time windows (bottom). Additional examples are provided at the end of this document. Additional examples and the code to reproduce all of the examples shown are provided in the gallery.

Getting Started

Installing or Upgrading hvsrpy

1. If you do not have Python 3.8 or later installed, you will need to do so. A detailed set of instructions can be found here.

2. If you have not installed hvsrpy previously, you can install it with pip via the command pip install hvsrpy. If you are not familiar with pip, a useful tutorial can be found here. If you have an earlier version and would like to upgrade to the latest version of hvsrpy use pip install hvsrpy --upgrade.

3. Confirm that hvsrpy has installed/updated successfully by examining the last few lines of the text displayed in the console.

Using hvsrpy

1. Download the contents of the examples directory to any location of your choice.

2. Launch the Jupyter notebook (example_mhvsr_traditional.ipynb) in the examples directory for a no-coding-required introduction to the basics of the hvsrpy package. If you have not installed Jupyter Lab, detailed instructions can be found here.

3. Continue to explore the other example notebooks provided. Note that each example notebook seeks to highlight one or two key features of hvsrpy and it may be helpful to combine multiple components together to build a custom workflow, this is highly encouraged.

4. Enjoy!

Looking for more information

More information regarding HVSR processing and hvsrpy can be found here and in the softwares documentation here.

Additional Comparisons between hvsrpy and Geopsy

Multiple Windows

The examples in this section use the same settings applied to different noise records. The settings are provided in the Settings section and the name of each file is provided above the corresponding figure in the Results section. The noise records (i.e., .miniseed files) are provided in the examples directory and also as part of a large published data set (Cox and Vantassel, 2018).

Settings

• Window Length: 60 seconds
• Bandpass Filter: No
• Tukey Window Length: 10% (i.e., 5% in Geopsy)
• Konno and Ohmachi Smoothing Coefficient: 40
• Resampling Center Frequencies:
  • Minimum: 0.3 Hz
  • Maximum: 40 Hz
  • Number: 2048
  • Type: 'log'
• Method for Combining Horizontal Components: 'squared-average'
• Distribution for f0 from Time Windows: 'normal'
• Distribution for Mean Curve: 'lognormal'

Multiple Window Results

File Name: UT.STN11.A2_C50.miniseed

File Name: UT.STN11.A2_C150.miniseed

File Name: UT.STN12.A2_C50.miniseed

File Name: UT.STN12.A2_C150.miniseed

Single Window

The examples in this section apply different settings to the same noise record (UT.STN11.A2_C50.miniseed). For brevity, the default settings are listed in the Default Settings section, with only the variations from these settings noted for each example.

Default Settings

• Window Length: 60 seconds
• Bandpass Filter: No
• Tukey Window Length: 10% (i.e., 5% in Geopsy)
• Konno and Ohmachi Smoothing Coefficient: 40
• Resampling Center Frequencies:
  • Minimum: 0.3 Hz
  • Maximum: 40 Hz
  • Number: 2048
  • Type: 'log'
• Method for Combining Horizontal Components: 'squared-average'
• Distribution for f0 from Time Windows: 'normal'
• Distribution for Mean Curve: 'lognormal'

Single Window Results

Default Case: No variation from those settings listed above.

Window Length: 120 seconds.

Cosine Taper Width: 20 % (i.e., 10 % in Geopsy)

Cosine Taper Width: 0.2 % (i.e., 0.1 % in Geopsy)

Konno and Ohmachi Smoothing Coefficient: 10

Konno and Ohmachi Smoothing Coefficient: 80

Number of Points: 512

Number of Points: 4096

hvsrpy's Issues

hvsrpy Community Survey

Dear hvsrpy Community,
It is hard to believe but hvsrpy (Vantassel, 2020) has just turned three. Over that time hvsrpy has seen widespread use in the fields of seismology, geophysics, and engineering and a rapidly growing userbase (over 30k downloads & over 45 stars on GitHub). As such, I have decided to undertake the first major overhaul of the hvsrpy codebase to add new features and streamline its API. Throughout this process I am looking for feedback (via this survey https://forms.gle/36aWUKrgGwiYddnSA) to understand how the community currently processes HVSR data and what new features the community would like to see in the future. Your responses are greatly appreciated. Keep an eye out for the release of hvsrpy v2.0.0 later this year.

Combine Three One-Component Miniseed Files to a Single Three-Component File

Problem Summary

Some seismic testing instruments save their measurements into three one-component miniseed files (one for north, east, and vertical). hvsrpy and the associated web-application hvsrweb assume a three-component miniseed file (i.e., all three traces are saved in the same miniseed file) by default. While Sensor3c's .from_mseed classmethod allows for the provision of three one-component files in lieu of a single three-component file (see docs for details), this feature has not yet been added to hsvrweb and remains a frequent point of inquiry.

Proposed Solution

Fortunately, combining three one-component miniseed files into a single three-component miniseed file is quite straightforward using obspy. I have posted code demonstrating how to do this as a gist here. I hope that if you are reading this you find it helpful.

CPU number for processing

Hi,

First of all, thanks for your efforts for HVSRpy.

I'm currently exploring the use of multiprocessing with HVSRpy.
However, I've noticed that the processing utilizes all available CPU resources extensively.
Is there an option to limit the number of CPUs for a single process, similar to the -j option in geopsy?

Thank you,
segu

API Change Request: Multi-Azimuth Output File

API Change Request: Multi-Azimuth Output File

On behalf of Alan Thorp, Ground Investigation Ltd.

Problem

The output file created when considering multiple azimuths (i.e., via a HvsrRotated object) is comma delimited. However, many of the metadata parameters (e.g., LMf0,AZ and SigmaLNf0,AZ) contain commas, making it difficult to parse the files programmatically.

Proposed Solution

Remove the offending commas such that LMf0,AZ and SigmaLNf0,AZ become LMf0AZ and SigmaLNf0AZ, respectively.

(Continuous error): ValueError: illegal value in 4-th argument of internal None

To anyone who reads this,

I recorded ambient noise in a test site, and I am using hvsrpy to estimate f0 and a0. I had 4 sensors per site and the first two worked perfectly. However, when I tried to insert the miniseed of the third station, I always get the error "ValueError: illegal value in 4-th argument of internal None". I am attaching both the miniseed that worked and the miniseed that pops the error. This work is part of my PhD thesis, and I would gladly accept any assistance! The waveforms are here for 30 days: https://easyupload.io/m/r8n6ms

Thank you again,
Angelos Zymvragakis, MSc Geologist - Seismologist

Feature Request: Command line interface

It would be nice to be able to run the program from the command line for a list of files. This is the way that I have implemented by own HVSR calculation software. The following is an example of my command line interface:

usage: H/V Calculator [-h] [--version] [-a] [-c CONFIG] [-m {both,welch,sesame,raw_only}]
                      [-p {auto,manual}]
                      [inputs [inputs ...]]

Calculator for V/H ratios from mini-seed

positional arguments:
  inputs                Input file(s) to read

optional arguments:
  -h, --help            show this help message and exit
  --version             show program's version number and exit
  -a, --all             Compute all usable
  -c CONFIG, --config CONFIG
                        Specify configuration
  -m {both,welch,sesame,raw_only}, --method {both,welch,sesame,raw_only}
                        Method for computing H/V ratio
  -p {auto,manual}, --peakselect {auto,manual}
                        Method for selecting the peak

A few things:

• I would also add an argument for the output directory, and then a sub-directory would be based on the input file name.
• In each output directory, standard plots and output files are created.
• A configuration file can be specified so that an analysis is repeated. The configuration is also saved to the output to document that calculation.

I think such an interface would help in bulk processing of recordings. Additionally, interactively selecting frequency ranges could be done using matplotlib GUI support.

Supporting alternate data formats (i.e., not only miniSEED)

Problem Summary

Researchers use various equipment to measure ambient noise for HVSR, and this variety of equipment unfortunately results in a variety of data formats. Ideally hvsrpy will allow for convenient handling of a variety of common data formats and not only miniSEED.

Proposed Solution

As most researchers are able to convert their data to ASCII/UTF-8 characters it makes sense to extend hvsrpy to include that functionality as a first step. However, as the format of any text file may vary, its difficult to produce a single script that will extract the data appropriately. Therefore, it is important to keep in mind that the example provided below is in fact only an example of a potential solution that can/must be modified appropriately.

Examples

For MiniShark:

# Load metadata
with open(fname, "r") as f:
    lines = f.readlines()
for line in lines:
    if line.startswith("#Sample rate (sps):"):
        _, sample_rate = line.split(":\t")
sample_rate = float(sample_rate)
dt = 1/sample_rate

# Load data
keys = ["vt", "ew", "ns"]
df = pd.read_csv(file_name, comment="#", sep="\t", names=keys)
components= {key:sigpropy.TimeSeries(df[key], dt) for key in keys}

# Create Sensor3c object to replace hvsrpy.Sensor3c.from_mseed() 
sensor = hvsrpy.Sensor3c(**components, meta={"File Name": file_name})

For SESAME ASCII data format (SAF) v1

fname = "MT_20211122_133110.SAF"

with open(fname, "r") as f:
    lines = f.readlines()
    
for idx, line in enumerate(lines):
    if line.startswith("SAMP_FREQ = "):
        fs = float(line[len("SAMP_FREQ = "):])
    if line.startswith("####--------"):
        idx += 1
        break

vt = []
ns = []
ew = []
for line in lines[idx:]:
    _vt, _ns, _ew = line.split()
    vt.append(_vt)
    ns.append(_ns)
    ew.append(_ew)

vt = sigpropy.TimeSeries(vt, dt=1/fs)
ns = sigpropy.TimeSeries(ns, dt=1/fs)
ew = sigpropy.TimeSeries(ew, dt=1/fs)
    
sensor = hvsrpy.Sensor3c(ns, ew, vt)

Pre-processing?

Hi Joseph,

Thanks for putting together this amazing package!

I'm doing some initial exploration with a raw data set collected from East Antarctica and encountered some 'interesting' spectra:

Being a relative beginner with HVSR processing, I wanted to ask what your recommended data pre-processing steps are before HVSR analysis in hvsrpy, beyond the implemented time-domain filtering and spectral smoothing? I understand instrumental response correction may not be necessary if responses across components are similar, but can, e.g., detrending, static shift corrections be implemented in hvsrpy? What are the typical steps you take? I'd be grateful for any advice you could give!

Cheers,

Ian

Clarity criteria don't respect the frequency filter range

Hi,

It seems that the clarity criteria are not drawing their statistics from between f_range_low and f_range_high.

For example:
Criteria iii): Pass
A0[f0mc]=3.017 is > 2.0

I got this result when using 3 Hz to 5 Hz. You can see in the image below that this amplitude is almost certainly coming from the large peak that I'm trying to avoid.

I haven't check the other criteria so it may be that only the amplitude is an issue.

Output final results

In the previous version it was possible to save the final results by
hv. to_file (file_name_out, distribution_f0, distribution_mc, data_format= "hvsrpy").
Is there a way to do the same in this new version?
Thank you

Problem reading SAF v1 format file

Hi,
first af all, thank you for sharing you work with the community.
I am writing this post because I am having problem to read in hvsrpy SAF ASCI format file (v.1).

I have uploaded here an example file (obtained exporting the trace, which has been registered with Tromino (Moho), from the Grilla software).
EqualizedFile-HV01.zip

In the following I have pasted the error I get.
As far as I can understand, it looks like, the 'type' is not recognized correctly; it appears to identify it as type equal to 'peer'

Thank you again
Giuseppe

" .... HV_001part1/test5_001.saf does not include the NORTH_ROT keyword, assuming equal to zero.
warnings.warn(msg, UserWarning)

ValueError Traceback (most recent call last)
Cell In[9], line 1
----> 1 srecords = hvsrpy.read(fnames)
2 srecords = hvsrpy.preprocess(srecords, preprocessing_settings)
3 hvsr = hvsrpy.process(srecords, processing_settings)

File ~\anaconda3\envs\HVSR\Lib\site-packages\hvsrpy\data_wrangler.py:701, in read(fnames, obspy_read_kwargs, degrees_from_north)
698 if len(fname) == 1:
699 fname = fname[0]
--> 701 seismic_recordings.append(read_single(fname,
702 obspy_read_kwargs=read_kwargs,
703 degrees_from_north=degrees_from_north))
705 return seismic_recordings

File ~\anaconda3\envs\HVSR\Lib\site-packages\hvsrpy\data_wrangler.py:619, in read_single(fnames, obspy_read_kwargs, degrees_from_north)
616 logger.info(f"Tried reading as {ftype}, got exception | {e}")
618 if ftype == "peer":
--> 619 raise e
621 pass
622 else:

File ~\anaconda3\envs\HVSR\Lib\site-packages\hvsrpy\data_wrangler.py:612, in read_single(fnames, obspy_read_kwargs, degrees_from_north)
610 for ftype, read_function in READ_FUNCTION_DICT.items():
611 try:
--> 612 srecording_3c = read_function(fnames,
613 obspy_read_kwargs=obspy_read_kwargs,
614 degrees_from_north=degrees_from_north)
615 except Exception as e:
616 logger.info(f"Tried reading as {ftype}, got exception | {e}")

File ~\anaconda3\envs\HVSR\Lib\site-packages\hvsrpy\data_wrangler.py:468, in _read_peer(fnames, obspy_read_kwargs, degrees_from_north)
466 msg = "Must provide 3 peer files (one per trace) as list or tuple, "
467 msg += f"not {type(fnames)}."
--> 468 raise ValueError(msg)
470 component_list = []
471 component_keys = []

ValueError: Must provide 3 peer files (one per trace) as list or tuple, not <class 'str'>. "

Problems reading .miniseed files

Hello!

My name is David. First of all, thank you very much for this code, it looks really great and very useful!

Well, lets go to the issue. I'm having some problems when I use my own .miniseed files with your codes. For example, i'm struggling with the code named "Gallery of mHVSR Examples Automatically Checked with SESAME (2004) Reliability and Clarity Criteria". When I use my .miniseed file (or even yours), I obtain the same problem message:

ValueError: Must provide 3 peer files (one per trace) as list or tuple, not <class 'str'>

I thought that maybe the problem is because I merged the three components (E, N, Z) into 1 .miniseed file, but if I use it in Geopsy there's no problem, I can see each component and the obtain the HV curve. Even if I try to use the three .miniseed components I have problems.

I have to say that I'm still a beginner playing with python and I believe that I'm missing something.

Could you help me? Thank you in advance! ;)

Missing Azimuthal Statistics in `azimuthal_hvsrpy_interface.ipynb` (v0.4.1)

The Problem

The table showing the azimuthal HVSR statistics in azimuthal_hvsrpy_interface.ipynb is missing. The notebook produces no error.

Specific Problem

The method HvsrRotated.print_stats() is empty. Problem only exists in v0.4.1.

Fix

Reimplement the HvsrRotated.print_stats() method.

Issue with installing hvsrpy

Hey Joe,

I ma having some issue installing hvsrpy on my Jupiter lab. I run it via anaconda navigator to manage my environments and I am trying to install it on my obspy environment. The traceback/installation error I get when I execute "!pip install hvsrpy" is attached

issue installing hvsrpy.docx