egemenimre / ccsds-ndm

CCSDS Navigation Data Messages Read/Write Library

CCSDS "Navigation Data Messages" Read and Write

Description

The Consultative Committee for Space Data Systems (CCSDS) develops communications and data systems standards for spaceflight. CCSDS Navigation Data Messages (NDM) is the set of file standards to define common data types such as trajectory, orbit, attitude and conjunction events. These data types are routinely generated and exchanged within and between spacecraft operators, space agencies, researchers, amateurs and commercial companies. As such, accurate definition and common interpretation of the data is crucial (and sometimes mission-critical).

The standard description for each data type is encapsulated in an XML Schema file. This project ccsds-ndm aims to be the reference open-source Python implementation to read and write the NDM XML and KVN files, through an object tree API, auto-generated by these schema files. It supports the up-to-date NDM XML 2.0.0 standard.

The source code is on Github and the documentation is on Readthedocs.

Current functionality:

	Read	Write
XML	All NDM Types	All NDM Types
KVN	All except NDM Combined Instantiation	All except NDM Combined Instantiation
JSON	Not specified in CCSDS Standards	Not specified in CCSDS Standards

Usage and Examples

There are two use cases:

• The ccsds-ndm library reads the NDM file, fills an object tree and offers it to the users. The users will then have to fill their own attitude, orbit or trajectory objects used in their libraries.
• The user fills an object tree from their own attitude, orbit or trajectory object. The ccsds-ndm library writes the NDM file using this object tree.

For the first use case, reading an OEM file from xml_read_path is as simple as:

>>> cdm = NdmIo().from_path(xml_read_path)

Note that file format (XML or KVN) and data type (e.g. CDM or NDM) are inferred automatically. The output cdm is the object tree for a Conjunction Data Message (CDM). The contents can then be reached going deeper in the object tree as specified in the corresponding NDM Standard. This example shows how to reach the orbit normal position component of the relative state vector:

>>> print(cdm.body.relative_metadata_data.relative_state_vector.relative_position_n)

The data can sometimes be of type NDM Combined Instantiation. This means that there are multiple NDM data bodies (e.g. 2 AEMs, 3 OEMs and one OMM) within a single file. The file reader supports these types as well and they are kept within the Ndm object as individual lists for each of the file types. The following example retrieves the second Omm object in the NDM file and then continues to dive deeper into the object tree to retrieve the eccentricity value.

>>> print(ndm.omm[1].body.segment.data.mean_elements.eccentricity)

If the file is of the type NDM Combined Instantiation but there is only a single data (e.g. OMM) in it, the ndm tags are stripped and only the single data is presented to the user.

Filling the objects with data properly requires some care. As the standard is understandably strict, the object tree derived from the XSD files are also rather exacting in how they accept data. The recommended way to fill the objects is with value and unit information:

>>> # This is the proper way to write data into the object
>>> cdm.body.relative_metadata_data.relative_state_vector.relative_position_t = LengthType(Decimal(800), LengthUnits.M)

Finally, once filled with the relevant data, the cdm object can be written to xml_write_path in XML data format as:

>>> NdmIo().to_file(cdm, NDMFileFormats.XML, xml_write_path)

The ndm object trees are not very user friendly and most probably will have to be filled by the users' own equivalent objects (trajectory, orbit, attitude etc.).

Installing ccsds-ndm

The ccsds-ndm package is on PyPI and you can install it simply by running:

pip install ccsds_ndm

You can also install it via conda-forge:

conda install -c conda-forge ccsds_ndm

Do not install ccsds-ndm using sudo.

More Information Regarding the CCSDS-NDM

The top-level description of the standard is given in the Navigation Data — Definitions and Conventions Green Book and Navigation Data Messages Overview Green Book. Individual data types are defined in their individual definitions (e.g. Conjunction Data Message and Orbit Data Message). The centre for all the standards are CCSDS Mission Operations and Information Management Services Area.

The Schema files are found in the SANA Registry.

Design and Limitations

The object tree is created by xsdata library, which also handles parsing and writing of the XML data. As such, there is no documentation generated for this object tree.

File read is usually fast (on the order of seconds) for small files. That said, KVN parsing for large files can take some time. A 12 MB OEM file lasts about a minute to parse on a good consumer grade computer. Due to the fragility of the KVN format and the restrictions the standards put on the order of keys, no parallelization has been attempted.

Requirements

• xsData is used to read and write XML files (and also to generate the object tree)
• lxml to support XML object creation

Citation

Please use the DOI for citations. This is the latest version:

License

This project is Copyright (c) Egemen Imre and licensed under the terms of the GNU GPL v3+ license.

ccsds-ndm's Issues

Read the Docs - ilk adım

Read the Docs konusunda deneyimi olan arkadaşlar varsa buyrun söhpete. :)

Ben Read the Docs kullanıyorum ve memnunum, config dosyaları ve kurulum konusunda da ilk adımı atabilirim. Ama deneyimli birinden yardım gelirse daha iyi olur tabii.

.idea dizini silinecek

Yanlışlıkla eklendi, silinmesi gerek.

xsdata switchable docstyle

xsdata changed the way it handles the docstyle. Check and update the docstyle if necessary.

xsdata init issue:
tefra/xsdata#318

xsdata Issue resolution:
tefra/xsdata#325

tdm.body.segment.data not loaded in kvn format

Hi,

I downloaded your project at the latest version 2.1 and I read a kvn file using the command

cdm = NdmIo().from_path(input_kvn_file)

and printing cmd I got

Tdm(header=TdmHeader(comment=[], creation_date='2016-09-30T12:48:03', originator='ESA', message_id=None), body=TdmBody(segment=[TdmSegment(metadata=TdmMetadata(comment=[], track_id=None, data_types=None, time_system=<TimeSystemType.UTC: 'UTC'>, start_time='2016-09-23T22:16:15.00', stop_time='2016-09-23T22:17:55.00', participant_1='ZIMLAT', participant_2='auto01', participant_3=None, participant_4=None, participant_5=None, mode=<ModeType.SEQUENTIAL: 'SEQUENTIAL'>, path='2,1', path_1=None, path_2=None, ephemeris_name_1=None, ephemeris_name_2=None, ephemeris_name_3=None, ephemeris_name_4=None, ephemeris_name_5=None, transmit_band=None, receive_band=None, turnaround_numerator=None, turnaround_denominator=None, timetag_ref=None, integration_interval=None, integration_ref=None, freq_offset=None, range_mode=None, range_modulus=None, range_units=None, angle_type=<AngleTypeType.RADEC: 'RADEC'>, reference_frame=<RefFrameType.EME2000: 'EME2000'>, interpolation=None, interpolation_degree=None, doppler_count_bias=None, doppler_count_scale=None, doppler_count_rollover=None, transmit_delay_1=None, transmit_delay_2=None, transmit_delay_3=None, transmit_delay_4=None, transmit_delay_5=None, receive_delay_1=None, receive_delay_2=None, receive_delay_3=None, receive_delay_4=None, receive_delay_5=None, data_quality=None, correction_angle_1=None, correction_angle_2=None, correction_doppler=None, correction_mag=None, correction_range=None, correction_rcs=None, correction_receive=None, correction_transmit=None, correction_aberration_yearly=None, correction_aberration_diurnal=None, corrections_applied=None), data=None)]), id='CCSDS_TDM_VERS', version='2.0')

In the input kvn file data section contains values. How can I solve it?

Regards.

Andrea Chessa

ndm_io: auto-detect file type potential fail with v2.0 files

Auto-detect file type currently use v1.0 classes. When adding v2.0 schemas and file types, check for the version as well and use proper (v1.0 or v2.0) class.

XMLschema ile dosya okuma sorunu

XML schema ile örnek bir dosya okuma kodu yazdım, bununla standarttan alınmış olan örnek CDM dosyasını okumaya çalışıyorum. Ancak açıkça bir namespace sorunu var ve şöyle bir hata alıyorum:

xmlschema.validators.exceptions.XMLSchemaChildrenValidationError: failed validating <Element 'cdm' at 0x7f9838239540> with XsdGroup(model='sequence', occurs=[1, 1]):

Reason: Unexpected child with tag 'header' at position 1. Tag ndm:header expected.

Yardımcı olabilecek olan var mıdır?

import os
from builtins import print
from pathlib import Path
from pprint import pprint

import xmlschema

if __name__ == '__main__':

    # print working directory
    print(f"current working directory is {os.getcwd()}.")

    # check file location
    xml_file_path = Path(os.getcwd(), "..", "sample_xml", "cdm_example_section4.xml")

    print(f"xml file path :  {xml_file_path.absolute()}")
    print(f"file exists   : {xml_file_path.exists()}")

    # define XSD file
    xsd_file_base_path = Path(os.getcwd(), "..", "..", "xsd_files", "ndmxml-1.0.C-schemas-qualified")
    ndm_master_xsd_path = Path(xsd_file_base_path, "ndmxml-1.0-master.xsd")

    xsd = xmlschema.XMLSchema(str(ndm_master_xsd_path))

    # read XML file
    print(f"XML is valid  : {xsd.is_valid(str(xml_file_path))}")
    # print(xsd.validate(str(xml_file_path)))
    pprint(xsd.to_dict(str(xml_file_path)))

Cannot read OMM from Celestrak

I tried downloading https://raw.githubusercontent.com/egemenimre/ccsds-ndm/main/ccsds_ndm/tests/data/ndmxml-1.0-omm-2.0.xml and reading it, successfully:

>>> import ccsds_ndm
>>> ccsds_ndm.__version__
'1.1'
>>> from ccsds_ndm.ndm_io import NdmIo
>>> from pathlib import Path
>>> NdmIo().from_path(Path("ndmxml-1.0-omm-2.0.xml"))
Omm(header=NdmHeader(comment=['THIS EXAMPLE CONFORMS TO FIGURE 4-2 IN 502.0-B-2'], creation_date='2007-065T16:00:00', originator='NOAA/USA'), body=OmmBody(segment=OmmSegment(metadata=OmmMetadata(comment=[], object_name='GOES-9', object_id='1995-025A', center_name='EARTH', ref_frame='TEME', ref_frame_epoch=None, time_system='UTC', mean_element_theory='TLE'), data=OmmData(comment=['USAF SGP4 IS THE ONLY PROPAGATOR THAT SHOULD BE USED FOR THIS DATA'], mean_elements=MeanElementsType(comment=[], epoch='2007-064T10:34:41.4264', semi_major_axis=None, mean_motion=RevType(value=Decimal('1.00273272'), units=None), eccentricity=Decimal('0.0005013'), inclination=InclinationType(value=Decimal('3.0539'), units=None), ra_of_asc_node=AngleType(value=Decimal('81.7939'), units=None), arg_of_pericenter=AngleType(value=Decimal('249.2363'), units=None), mean_anomaly=AngleType(value=Decimal('150.1602'), units=None), gm=GmType(value=Decimal('398600.8'), units=None)), spacecraft_parameters=None, tle_parameters=TleParametersType(comment=[], ephemeris_type=None, classification_type=None, norad_cat_id=23581, element_set_no=925, rev_at_epoch=4316, bstar=BStarType(value=Decimal('0.0001'), units=None), mean_motion_dot=DRevType(value=Decimal('-0.00000113'), units=None), mean_motion_ddot=DdRevType(value=Decimal('0.0'), units=None)), covariance_matrix=None, user_defined_parameters=UserDefinedType(comment=[], user_defined=[UserDefinedParameterType(value='xyz', parameter='ABC0'), UserDefinedParameterType(value='9', parameter='ABC1'), UserDefinedParameterType(value='xyz', parameter='ABC2'), UserDefinedParameterType(value='9', parameter='ABC3'), UserDefinedParameterType(value='xyz', parameter='ABC4'), UserDefinedParameterType(value='9', parameter='ABC5'), UserDefinedParameterType(value='xyz', parameter='ABC6'), UserDefinedParameterType(value='9', parameter='ABC7'), UserDefinedParameterType(value='xyz', parameter='ABC8'), UserDefinedParameterType(value='9', parameter='ABC9')])))), id='CCSDS_OMM_VERS', version='2.0')

However, it doesn't seem to work for https://celestrak.com/NORAD/elements/gp.php?CATNR=45018&FORMAT=XML:

$ wget "https://celestrak.com/NORAD/elements/gp.php?CATNR=45018&FORMAT=XML" -O celestrak_omm.xml -q
$ python -q
>>> from ccsds_ndm.ndm_io import NdmIo
>>> from pathlib import Path
>>> NdmIo().from_path(Path("celestrak_omm.xml"))
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/juanlu/.pyenv/versions/poliastro38/lib/python3.8/site-packages/ccsds_ndm/ndm_io.py", line 96, in from_path
    data_type = _NdmDataType.find_element(root.attrib.get("id")).clazz
AttributeError: 'NoneType' object has no attribute 'clazz'

XML headers

See here:
tefra/xsdata#320 (comment)

Check for updates to xsdata

Multi-line COMMENT in OEM partly put in wrong section

Input is from "Figure 5-1" of "CCSDS RECOMMENDED STANDARD FOR ORBIT DATA MESSAGES" (CCSDS 502.0-B-2 the "Blue book", Issue November 2009) https://public.ccsds.org/Pubs/502x0b2c1e2.pdf
The part that has problems:

META_STOP
COMMENT This file was produced by M.R. Somebody, MSOO NAV/JPL, 1996NOV 04. It is
COMMENT to be used for DSN scheduling purposes only.
1996-12-18T12:00:00.331 2789.619 -280.045 -1746.755 4.73372 -2.49586 -1.04195
...

to reproduce, line5 and 6 below show the current parse result which is wrong:

fnm = 'oem_fig5_1.kvn'
contents = readfile(fnm)
ndm = NdmIo().from_string(contents)
assert ndm.body.segment[0].metadata.comment == []
assert ndm.body.segment[0].data.comment == [
    'This file was produced by M.R. Somebody, MSOO NAV/JPL, 1996NOV 04. It is']
assert ndm.body.segment[1].metadata.comment == [
    'to be used for DSN scheduling purposes only.']
assert ndm.body.segment[1].data.comment == []
assert ndm.body.segment[2].metadata.comment == []
assert ndm.body.segment[2].data.comment == [
        'This block begins after trajectory correction maneuver TCM-3.']

this should pass but is getting an AssertionError:

assert ndm.body.segment[0].data.comment == [
        'This file was produced by M.R. Somebody, MSOO NAV/JPL, 1996NOV 04. It is',
        'to be used for DSN scheduling purposes only.']

Released version does not contain source code

Hi! 👋 (Coming from https://twitter.com/poliastro_py/status/1331735972883664898)

I tried pip install ccsds_ndm as explained in the README, but could not import any code. I realized that the wheel available from PyPI is almost empty:

$ unzip -l ccsds_ndm-1.0-py2.py3-none-any.whl 
Archive:  ccsds_ndm-1.0-py2.py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
      210  2020-11-25 00:08   ccsds_ndm.py
    35149  2020-11-15 08:03   ccsds_ndm-1.0.dist-info/LICENSE
       99  2016-01-01 00:00   ccsds_ndm-1.0.dist-info/WHEEL
     5862  2016-01-01 00:00   ccsds_ndm-1.0.dist-info/METADATA
      363  2016-01-01 00:00   ccsds_ndm-1.0.dist-info/RECORD
---------                     -------
    41683                     5 files
$ cat ccsds_ndm.py 
# CCSDS-NDM: CCSDS Navigation Data Messages Read/Write Library
#
# Copyright (C) 2020 CCSDS-NDM Egemen Imre
#
# Licensed under GNU GPL v3.0. See LICENSE.rst for more info.
"""
CCSDS-NDM
"""
__version__ = "1.0"

So probably there is some sort of packaging problem.

missing requirement to ```lxml```

If ccsds_ndm is installed with pip install ccsds_ndm as shown in README.rst, lxml does not get installed. requirements.txt does include lxml, though.

Cannot write KVN file (`'Ndm' object has no attribute 'id'`)

I tried the following:

ndm = NdmIo().from_path("omm_combined.xml")  # omm_combined.xml from the repository
NdmIo().to_file(ndm, NDMFileFormats.KVN, Path("new.kvn"))

but it failed with this error:

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-23-60823d5a4046> in <module>
----> 1 NdmIo().to_file(ndm, NDMFileFormats.KVN, Path("new.kvn"))

~/Projects/LSF/opensatcom/constellation-designer/.venv/lib/python3.9/site-packages/ccsds_ndm/ndm_io.py in to_file(self, ndm_obj, data_format, xml_write_file_path, **kwargs)
    158 
    159         if data_format is NDMFileFormats.KVN:
--> 160             return NdmKvnIo().to_file(ndm_obj, xml_write_file_path)
    161 
    162         if data_format is NDMFileFormats.JSON:

~/Projects/LSF/opensatcom/constellation-designer/.venv/lib/python3.9/site-packages/ccsds_ndm/ndm_kvn_io.py in to_file(self, ndm_obj, kvn_write_file_path)
    226             Path of the XML file to be written
    227         """
--> 228         kvn_write_file_path.write_text(self.to_string(ndm_obj))
    229 
    230     def to_string(self, ndm_obj):

~/Projects/LSF/opensatcom/constellation-designer/.venv/lib/python3.9/site-packages/ccsds_ndm/ndm_kvn_io.py in to_string(self, ndm_obj)
    242             given object tree as KVN string
    243         """
--> 244         out_str = [_fill_str_out_kvn(ndm_obj.id, ndm_obj.version)]
    245         self._collate_str_out("", ndm_obj, out_str)
    246 

AttributeError: 'Ndm' object has no attribute 'id'

Am I doing anything wrong?

How to handle multi-NDMs?

Normally NDM files are "single file-single data", however Combined Instantiation (or multi-NDMs in our parlance) do not conform to this rule. Thinking about how to set up the interface for the user...

• A multi_ndm object, as opposed to an (single-)ndm object?
• Giving the multi-ndm object to the user as normal?
• Helping the user when the multi-ndm is actually a single-ndm?

discussion moved from the "OMM read error" issue, which is now solved: #10

Continuous Integration - ilk adım

Continuous integration konusunda deneyimi olan arkadaşlar varsa buyrun söhpete. :)

Ben Circle CI kullanıyorum ve memnunum, config dosyaları ve kurulum konusunda da ilk adımı atabilirim. Ama deneyimli birinden yardım gelirse daha iyi olur tabii.