datastreamapp / schema Goto Github PK

The hyperlink under the "Dataset Metadata" section of the ReadMe does not redirect to an existing webpage.

Use ajv-pack

Remove current logic for ResultSampleFraction warning:
https://github.com/gordonfn/schema/blob/main/schemas/data/src/quality-control/CharacteristicName-ResultSampleFraction.json

Replace with warning for ResultSampleFraction when ActivityType is Field Msr/Obs or Field Msr/Obs-Portable Data Logger

• https://json-schema.org/draft/2019-09/release-notes.html
  • dependencies -> dependentRequired
• ajv support - ajv-validator/ajv#1082

trim values from google sheet

Determine naming convention for mapping stable isotope characteristics.

• Review existing common stable isotope characteristic names in WQX (e.g. 18-O, 15-N, 2-H)
• Create guidance/convention for mapping stable isotopes:
• Characteristic name can be the isotope name (e.g. Deuterium) or the isotope as delta notation (d18O)
• Use ResultUnit to specify how it was measured (e.g. as a ratio, per mil)
• Need field to enter what molecule isotope is being measured in (e.g. d180 of water, d18O of nitrate, d15N of nitrate, d15N of ammonia, etc.)
• Remove ambiguous characteristic names

Currently, side-loading until PR merged ajv-validator/ajv-keywords#64

cd-ing to pkg and running npm i && npm run build is required in intrum

When ActivityStartTime, ActivityEndTime, or AnalysisStartTime are missing a leading zero in the 00:00:00 format, warning message is not being generated.

Ref: ajv-validator/ajv-keywords#64

http://www.unece.org/uncefact/codelistrecs.html Ref 20

Attached file contains proposed definitions for units in the additions lis
definition_result_unit_additions.csv
t

e.g. C4-Fluorenes, Thorium

Document process for adding new characteristics and method speciations. add it to the readme.

See https://github.com/gordonfn/schema/blob/master/src/addition/CharacteristicName.json and submit to https://www.epa.gov/waterdata/storage-and-retrieval-and-water-quality-exchange-domain-services-and-downloads#domain

Meeting set for Jan 22

When a result value and a result detection condition are both present the current schema issues a OneOf error. Could this be changed to better reflect the actual error?

New Text Example:

Only one of ResultValue or ResultDetectionConditon may exist

allows for private locations

unit assumed to be m

Enforcing the suggested logic will add a complexity and decrease maintainability

Options:

• make optional
• required but set a default
• add Not Applicable option to list and use as default
• add it to the schema

Based on feedback, we will change to Date & Time and infer timezone from lat/lng.

lat/lng -> timezone https://stackoverflow.com/questions/16086962/how-to-get-a-time-zone-from-a-location-using-latitude-and-longitude-coordinates/16086964

For ResultValue-TemperatureRange quality control check:
When temperature is outside expected range (-100 - 100), Message is should be >= 0 .
Can this be changed to: should be >= -100 & <= 100

• fix bug
• test

Use: ajv-istanbul

Need to add schema check to prevent the addition of a detection limit measure when not detected or detected not quantified are inputted

If (ResultDetectionCondition == 'Not Detected' or ReseultDetectionCondition == 'Detected Not Quantified') and ResultDetectionLimitMeasure != NULL:
return (Error Message saying it has to be NULL)

Effected Parameters:

• Ammonia
• Ammonium
• Inorganic nitrogen ammonia, nitrate and nitrite)
• Inorganic nitrogen nitrate and nitrite)
• Kjeldahl nitrogen
• Nitrate
• Nitrite
• Total nitrogen, mixed forms
• Organic Nitrogen
• Organic Phosphorus
• Orthophosphate
• Total phosphorus, mixed forms
• Soluble Reactive Phosphorus (SRP)

Acceptable Sample Fractions:

• Filtered
• Unfiltered
• Filtered, lab
• Filtered, field
• Non-filterable

We should give a warning if the sample fractions for these parameters don't match these. This is a very common issue in DataStream.

Nutrient Guidance

• remove enumCase transform (this would disable support for autocorrect for mixed-case values) - 2021-02 not at this time, still valuable to contributors
• reduce CharacteristicName list size
• reduce ResultUnit list size

Character limit for LaboratoryName is currently set to 60 characters. Limit should be increased to 100 characters.

• reach out to EPA - update in to WQX
• update db
• update schema
• deploy

hopefully, this is the right spot to request this, but i noticed when exporting data it doesn't follow tidy principles in that the variable 'Characteristic Name' contains many different types of observations. This makes it difficult to use the data cleanly in a GIS environment as a .csv and requires pre-processing.

• When ActivityType =
  "Quality Control Sample-Lab Blank",
  "Quality Control Sample-Lab Matrix Spike",
  "Quality Control Sample-Lab Spike",
  "Quality Control Sample-Reagent Blank",
  "Quality Control Sample-Reference Sample",
  "Quality Control Sample-Trip Blank",
  MonitoringLocation metadata is not required.
• When ActivityType = "Quality Control Sample-Trip Blank", ActivityStartDate required, else date recorded only under AnalysisStartDate.
• Observations joined based on DatasetName