Handling of UNDEF about fmu-dataio

[perolavsvendsen]

fmu-dataio/src/fmu/dataio/_export_item.py

Line 393 in ff35c42

meta["spec"]["undef"] = "Nan"

[perolavsvendsen]

Specific symptom addressed temporarily through #41 but root cause is probably something that must be discussed further.

undef is different in the context of datatypes, formats and contents (e.g. an Irap Binary surface has pre-defined undef in its format - a float value. In the current schema, the data.spec.undef attribute states the undefined value from a content perspective.

Example: From the metadata, the client should understand that the value -999.25 should be read as null.

This raises also a challenge in situations where null is the actual "value" to be used (e.g. for a specific content and a specific data type, None/null is actually the undefined marker. Using null is also ambiguous and should probably be avoided. There may also be incompatibility here with existing functionality for stripping away attributes with null values before dumping to yaml. In that context, a string (such as the "Nan" that was originally used) could be a solution, but it would be nice to avoid type-mixing for this attribute.

Perhaps -999.25 is the best option in many cases where null is not directly present, and perhaps null can be assumed when the data.spec.undef field is not present? (In Python, use .get("undef")) This way, the field - if present, will always be a number.