eurec4a / flight-phase-separation Goto Github PK

Collection of manually edited flight segments for all platforms participating in EUREC4A.

Python 84.77% HTML 15.23%

flight-phase-separation's Introduction

EUREC4A meta data repository

Shared metadata, code and standards designed to structure the EUREC4A data reposistory, improve treatment of metadata and thereby ease the EUREC4A data analysis.

State of the project

Currently, the goals of the metadata repository are being refined while experiments on fist metadata structures, mostly considering measurement platforms are conducted. If you just came across this repository, please have a look at the current goals document as well as issues and pull requests. Please don't hesitate to add more issues or pull requests if you have use cases, expectations, opinions or questions about EUREC4A data or metadata, which are not yet written down.

Metadata concept

EUREC4A metadata will be sourced from the owners of the objects the metadata describes. That is each instrument would provide instrument metadata, with a minimal set of controlled language, and a subset of this infomation would be inherited by the platform metadata. Campaign metadata can then inherit information from the plafforms. This ensures that all the metadata is provided by the owners. At each stage of the process additional information could be included an inherited. An example would be the flight-track dictionaries being developed for HALO, which would then find their way into the HALO metadata.

Metadata also proide a controlled vocabulary i.e. a list of valid ways to refer to a platform, instrument, etc.

Tentiative naming conventions

Although file naming conventions are not a particular goal of this metadata concept, some ideas have already evolved and can be found in naming_conventions.md for reference.

flight-phase-separation's People

Contributors

Stargazers

Watchers

Forkers

eurec4a-uk geet-george heikekonow leifdenby maahn tmieslinger

flight-phase-separation's Issues

SAM error

Dear Geet
I just spotted a mistake in the yaml file:
Sonde "HALO-0211_s60" shouldn't be part of circle 5 on the 11 Feb - it was dropped 2º east of circle east and 20min after the circle. Can you change that directly or should I ask Marc?
Cheers, raphaela

Better sorting for compiled yaml keys

Currently the keys in the compiled all_flights.yaml are sorted alphabetically. We might want to transition to a more human readable order.

Sonde overlap among segments?

For the HALO flight on 15th Feb, the last 2 circles were planned so that 1 sonde was an overlap between the two consecutively flown dropsonde-circles (i.e. no usual break of 15 minutes).

Currently, the flight phase separation (FPS) files classify that single sonde in to one of the circles. However, to make the JOANNE circle products robust, we need this sonde to be present in both circles, i.e. Sonde HALO-0215_s44 should be included in both HALO-0215_c5 and HALO-0215_c6.

For JOANNE, I am also making a sonde-2-circle reference list for Level-4, to identify the sondes present in the circles. Ideally, I'd like this to also take input straight from the FPS files.

So, even if it is for just one sonde, it'll be nice to allow sondes to belong to multiple segments. Do we have any rules (or do we want any) stating that sondes should belong to only one segment?

YAML -> netCDF converter

YAML is not a convenient file format in all contexts. Users have expressed interest in a tool that converts YAML flight phase files to netCDF time series files. Roughly the idea would be to copy the time variable from some reference file, than create an array with dimensions (time, segment_kinds) with True/False values for each segment kind defined in the file. @d70-t points out CF conventions can be used to map arbitrary segment names to segment_kinds.

Initial automated reporting

It would be neat to ensure that each YAML file phase file is sensible. That might mean

segment times (other than ground) lie between take-off and landing times
only one primary kind per segment(?
??

It would also be neat to make light-weight diagnostics at each push. One might be plan and/or side views of the segments for each flight, organized in some sensible way.

@RobertPincus can set this up to run for each commit once scripts are complete.

sondes.yaml contains only HALO sondes

File scripts/sondes.yaml includes only the HALO sondes. I have a similar file from Geet - shall I add those, or keep a separate file and rename the existing one to halo_sondes.yaml?

Segment kinds

Would it be possible to rename the radar_calibration_wiggles rather radar_calibration_roll, and radar_calibration_tilted rather radar_calibratio_pearl. There reason is that the former is a roll maneuver, this is also common for calibrating the gust probes, and I would actually simply call this a roll maneuver. The pearl is short circle, sometimes longer, and is a meaningful maneuver even if not calibrating the radar. For tis reason one could be a bit more general and say: pearl, and roll maneuvers.

An example file looks like

name: RF03 
mission: EUREC4A
flight_id: ATR-0126 
contacts:
- name: Author
  email: [email protected]
date: 2020-01-26 
flight_report: https://observations.ipsl.fr/aeris/eurec4a-data/REPORTS/ATR-42/2020/20200126/ATR-0126.pdf 
takeoff: 2020-01-26 11:34:57 
landing: 2020-01-26 16:08:07 
events: []
remarks: 'synchronisation problem of Picarro with ATR core data, shifting during flight'
segments:
- kind: dry event
  name: D1
  start: 2020-01-26 15:46:00
  end: 2020-01-26 16:04:00

What would be needed to make these files available through the flight-segmentation API? Should the files be copied into this repo or could they be accessed from its current published location at https://observations.ipsl.fr/aeris/eurec4a-data/AIRCRAFT/ATR/vapour/ ?

HALO flight phase file names don't mention version number

To keep with EUREC4A's convention, we should add version numbers to the file names?