GithubHelp home page GithubHelp logo

samueljennings / mt_metadata Goto Github PK

View Code? Open in Web Editor NEW

This project forked from kujaku11/mt_metadata

0.0 0.0 0.0 40.28 MB

Tools for standardizing metadata, geared towards magnetotelluric (MT) data but is general enough to accommodate "any" type of metadata.

License: MIT License

Python 98.67% Objective-J 1.15% Makefile 0.18%

mt_metadata's Introduction

mt_metadata version 0.2.0

Standard MT metadata

PyPi version Latest conda|conda-forge version codecov example workflow name License: MIT DOI Binder

Description

MT Metadata is a project led by IRIS-PASSCAL MT Software working group and USGS to develop tools that standardize magnetotelluric metadata, well, at least create tools for standards that are generally accepted. This include the two main types of magnetotelluric data

  • Time Series

    • Structured as:
      • Experiment -> Survey -> Station -> Run -> Channel
    • Supports translation to/from StationXML
  • Transfer Functions

    • Supports (will support) to/from:
      • EDI (most common format)
      • ZMM (Egberts EMTF output)
      • JFILE (BIRRP output)
      • EMTFXML (Kelbert's format)
      • AVG (Zonge output)

Most people will be using the transfer functions, but a lot of that metadata comes from the time series metadata. This module supports both and has tried to make them more or less seamless to reduce complication.

  • Version: 0.2.0
  • Free software: MIT license
  • Documentation: https://mt-metadata.readthedocs.io.
  • Examples: Click the Binder badge above and Jupyter Notebook examples are in mt_metadata/examples/notebooks and docs/source/notebooks
  • Suggested Citation: Peacock, J. R., Kappler, K., Ronan, T., Heagy, L., Kelbert, A., Frassetto, A. (2022) MTH5: An archive and exchangeable data format for magnetotelluric time series data, Computers & Geoscience, 162, doi:10.1016/j.cageo.2022.105102

Installation

From Source

git clone git://github.com/kujaku11/mt_metadata

python setup.py install

You can add the flag -e if you want to change the code.

PIP

pip install mt_metadata

Conda

conda install mt_metadata

Standards

Each metadata keyword has an associated standard that goes with it. These are stored internally in JSON file. The JSON files are read in when the package is loaded to initialize the standards. Each keyword is described by:

  • type - How the value should be represented based on very basic types

    • string
    • number (float or integer)
    • boolean
  • required - A boolean (True or False) denoting whether the metadata key word required to represent the data.

  • style - How the value should be represented within the type. For instance is the value a controlled string where there are only a few options, or is the value a controlled naming convention where only a 5 character alpha-numeric string is allowed. The styles are

    • Alpha Numeric a string with alphabetic and numberic characters
    • Free Form a free form string
    • Controlled Vocabulary only certain values are allowed according to options
    • Date a date and/or time string in ISO format
    • Number a float or integer
    • Boolean the value can only be True or False
  • units - Units of the value

  • description - Full description of what the metadata key is meant to convey.

  • options - Any options of a Controlled Vocabulary style.

  • alias - Any aliases that may represent the same metadata key.

  • example - An example value to inform the user.

All input values are internally validated according to the definition providing a robust way to standardize metadata.

Each metadata object is based on a Base class that has methods:

  • to/from_json
  • to/from_xml
  • to_from_dict
  • attribute_information

And each object has a doc string that describes the standard:

Metadata Key Description Example
key description of what the key describes example value
Required: False
Units: None
Type: string
Style: controlled vocabulary

The time series module is more mature than the transfer function module at the moment, and this is still a work in progress.

Example

from mt_metadata import timeseries
x = timeseries.Instrument()

Help

help(x)

+----------------------------------------------+-----------------------------------------------+----------------+
| **Metadata Key**                             | **Description**                               | **Example**    |
+==============================================+===============================================+================+
| **id**                                       | instrument ID number can be serial number or  | mt01           |
|                                              | a designated ID                               |                |
| Required: True                               |                                               |                |
|                                              |                                               |                |
| Units: None                                  |                                               |                |
|                                              |                                               |                |
| Type: string                                 |                                               |                |
|                                              |                                               |                |
| Style: free form                             |                                               |                |
+----------------------------------------------+-----------------------------------------------+----------------+
| **manufacturer**                             | who manufactured the instrument               | mt gurus       |
|                                              |                                               |                |
| Required: True                               |                                               |                |
|                                              |                                               |                |
| Units: None                                  |                                               |                |
|                                              |                                               |                |
| Type: string                                 |                                               |                |
|                                              |                                               |                |
| Style: free form                             |                                               |                |
+----------------------------------------------+-----------------------------------------------+----------------+
| **type**                                     | instrument type                               | broadband      |
|                                              |                                               | 32-bit         |
| Required: True                               |                                               |                |
|                                              |                                               |                |
| Units: None                                  |                                               |                |
|                                              |                                               |                |
| Type: string                                 |                                               |                |
|                                              |                                               |                |
| Style: free form                             |                                               |                |
+----------------------------------------------+-----------------------------------------------+----------------+
| **model**                                    | model version of the instrument               | falcon5        |
|                                              |                                               |                |
| Required: False                              |                                               |                |
|                                              |                                               |                |
| Units: None                                  |                                               |                |
|                                              |                                               |                |
| Type: string                                 |                                               |                |
|                                              |                                               |                |
| Style: free form                             |                                               |                |
+----------------------------------------------+-----------------------------------------------+----------------+

Fill in metadata

x.model = "falcon 5"
x.type = "broadband 32-bit"
x.manufacturer = "MT Gurus"
x.id = "f176"

to JSON

print(x.to_json())
{
    "instrument": {
        "id": "f176",
        "manufacturer": "MT Gurus",
        "model": "falcon 5",
        "type": "broadband 32-bit"
    }
}

to XML

print(x.to_xml(string=True))
<?xml version="1.0" ?>
<instrument>
    <id>f176</id>
    <manufacturer>MT Gurus</manufacturer>
    <model>falcon 5</model>
    <type>broadband 32-bit</type>
</instrument>

Credits

This project is in cooperation with the Incorporated Research Institutes of Seismology, the U.S. Geological Survey, and other collaborators. Facilities of the IRIS Consortium are supported by the National Science Foundation’s Seismological Facilities for the Advancement of Geoscience (SAGE) Award under Cooperative Support Agreement EAR-1851048. USGS is partially funded through the Community for Data Integration and IMAGe through the Minerals Resources Program.

mt_metadata's People

Contributors

kkappler avatar kujaku11 avatar lheagy avatar

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.