GithubHelp home page GithubHelp logo

stanford-navlab / gnss_lib_py Goto Github PK

View Code? Open in Web Editor NEW
111.0 8.0 27.0 44.83 MB

Modular Python tool for parsing, analyzing, and visualizing Global Navigation Satellite Systems (GNSS) data and state estimates

License: MIT License

Python 84.01% Shell 0.19% Jupyter Notebook 15.81%
android gnss navigation position residual pseudorange ekf wls python

gnss_lib_py's Introduction

build codecov Documentation Status Open In Colab

gnss_lib_py

gnss_lib_py is a modular Python tool for parsing, analyzing, and visualizing Global Navigation Satellite Systems (GNSS) data and state estimates. It also provides an intuitive and modular framework allowing users to quickly prototype, implement, and visualize GNSS algorithms. gnss_lib_py is modular in the sense that multiple types of algorithms can be easily exchanged for each other and extendable in facilitating user-specific extensions of existing implementations.

satellite skyplot

gnss_lib_py contains parsers for common file types used for storing GNSS measurements, benchmark algorithms for processing measurements into state estimates and visualization tools for measurements and state estimates. The modularity of gnss_lib_py is made possibly by the unifying NavData class, which contains methods to add, remove and modify numeric and string data consistently. We provide standard row names for NavData elements on the reference page. These names ensure cross compatibility between different datasets and algorithms.

Documentation

Full documentation is available on our readthedocs website.

Code Organization

gnss_lib_py is organized as:

   ├── data/                          # Location for data files
      └── unit_test/                  # Data files for unit testing
   ├── dev/                           # Code users do not wish to commit
   ├── docs/                          # Documentation files
   ├── gnss_lib_py/                   # gnss_lib_py source files
        ├── algorithms/               # Navigation algorithms
        ├── navdata/                  # NavData data structure
        ├── parsers/                  # Data parsers
        ├── utils/                    # GNSS and common utilities
        ├── visualizations/           # plotting functions
        └── __init__.py
   ├── notebooks/                     # Interactive Jupyter notebooks
        ├── tutorials/                # Notebooks with tutorial code
   ├── results/                       # Location for result images/files
   ├── tests/                         # Tests for source files
      ├── algorithms/                 # Tests for files in algorithms
      ├── navdata/                    # Tests for files in navdata
      ├── parsers/                    # Tests for files in parsers
      ├── utils/                      # Tests for files in utils
      ├── visualizations/             # Tests for files in visualizations
      └── test_gnss_lib_py.py         # High level checks for repository
   ├── CONTRIBUTORS.md                # List of contributors
   ├── build_docs.sh                  # Bash script to build docs
   ├── poetry.lock                    # Poetry specific Lock file
   ├── pyproject.toml                 # List of package dependencies
   └── requirements.txt               # List of packages for pip install

In the directory organization above:

  • The algorithms directory contains localization algorithms that work by passing in a NavData class. Currently, the following algorithms are implemented in the algorithms:

    • Weighted Least Squares
    • Extended Kalman Filter
    • Calculating pseudorange residuals
    • Fault detection and exclusion
  • The data parsers in the parsers directory allow for either loading GNSS data into gnss_lib_py's unifying NavData class or parsing precise ephemerides data. Currently, the following datasets and types are supported:

  • The utils directory contains utilities used to handle GNSS measurements, time conversions, coordinate transformations, visualizations, calculating multi-GNSS satellite PVT information, satellite simulation, file operations, etc.

Installation

gnss_lib_py is available through pip installation with:

pip install gnss-lib-py

For directions on how to install an editable or developer installation of gnss_lib_py on Linux, MacOS, and Windows, please see the install instructions.

Tutorials

We have a range of tutorials on how to easily use this project. They can all be found in the tutorials section.

Reference

References on the package contents, explanation of the benefits of our custom NavData class, and function-level documentation can all be found in the reference section.

Contributing

If you have a bug report or would like to contribute to our repository, please follow the guide on the contributing page.

Troubleshooting

Answers to common questions can be found in the troubleshooting section.

Attribution

This project is a product of the Stanford NAV Lab and currently maintained by Ashwin Kanhere (akanhere [at] stanford [dot] edu) and Derek Knowles (dcknowles [at] stanford [dot] edu). If using this project in your own work please cite either of the following:


   @inproceedings{knowlesmodular2022,
      title = {A Modular and Extendable GNSS Python Library},
      author={Knowles, Derek and Kanhere, Ashwin V and Bhamidipati, Sriramya and Gao, Grace},
      booktitle={Proceedings of the 35th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2022)},
      institution = {Stanford University},
      year = {2022 [Online]},
      url = {https://github.com/Stanford-NavLab/gnss_lib_py},
      doi = {10.33012/2022.18525}
   }

   @inproceedings{knowles_kanhere_baselines_2023,
      title = {Localization and Fault Detection Baselines From an Open-Source Python GNSS Library},
      author={Knowles, Derek and Kanhere, Ashwin V and Gao, Grace},
      booktitle={Proceedings of the 36th International Technical Meeting of the Satellite Division of The Institute of Navigation (ION GNSS+ 2023)},
      institution = {Stanford University},
      year = {2023 [Online]},
      url = {https://github.com/Stanford-NavLab/gnss_lib_py},
   }

Additionally, we would like to thank all contributors to this project.

gnss_lib_py's People

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

gnss_lib_py's Issues

num_cols is not set on NavData.copy()

Describe the bug
When using "copy" to pull certain columns out of a larger "NavData" structure, the new NavData has not yet set the num_cols field appropriately. It says num_cols is 0 when there are values in there.

To Reproduce
This is some simple code to reproduce. I am using the Chemnitz data, I am attaching a CSV of just the first 20 lines or so to make this issue self-contained.

import gnss_lib_py as glp
import numpy as np

test_file = 'berlin1_potsdamer_platz/RXM-RAWX_small.csv'

nav_data_csv = glp.SmartLocRaw(test_file)
to_grab=np.arange(10)
small_nav_data = nav_data_csv.copy(cols=to_grab)
print('The copied one says it has ',small_nav_data.num_cols,'entries')
print('But it really has ',small_nav_data.array.shape[1],'entries')

Expected behavior
The two numbers should be equal.

RXM-RAWX_small.csv

Read RINEX Observation data failed!

Describe the bug
RinexObs("TWTF00TWN_R_20232490000_01D_30S_MO.rnx") run long time and get error report

/home/zsp/.local/lib/python3.8/site-packages/gnss_lib_py/utils/time_conversions.py:586: RuntimeWarning: No time zone info found in datetime, assuming UTC
  warnings.warn("No time zone info found in datetime, assuming UTC",\
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/zsp/.local/lib/python3.8/site-packages/gnss_lib_py/parsers/rinex_obs.py", line 83, in __init__
    obs_navdata_raw.find_wildcard_indexes(f'{measure_char}{band}*',
  File "/home/zsp/.local/lib/python3.8/site-packages/gnss_lib_py/parsers/navdata.py", line 943, in find_wildcard_indexes
    raise KeyError("More than " + str(max_allow) \
KeyError: 'More than 1 possible row indexes for C1*'

TWTF00TWN_R_20232490000_01D_30S_MO.rnx.gz

Refactor Python Code

Currently, the python code is scattered across different files in a way that doesn't make sense. Eg. calc_prange.py calculates the received pseudorange for an Android phone while expected pseudoranges are calculated in sim_gnss.py.
There is potentially a better way to arrange the functionality across different files

Colab installation requires older pandas

Describe the bug
Installing the library on Colab seems to require an older version of pandas

To Reproduce
Steps to reproduce the behavior:
Run pip install on Google Colab

Screenshots

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-colab 1.0.0 requires pandas==1.5.3, but you have pandas 2.0.3 which is incompatible.

importlib.metadata.PackageNotFoundError: No package metadata was found for gnss-lib-py

Describe the bug
"poetry run pytest" reports package metadata not found

To Reproduce
Steps to reproduce the behavior:

  1. git clones the gnss_lib_py repository;
  2. run poetry run pytest

Expected behavior
Could execute the test scripts

Screenshots
[mikehuang@localhost gnss_lib_py-1.0.2]$ poetry run pytest
Creating virtualenv gnss-lib-py-7gV7y9ay-py3.11 in /home/mikehuang/.cache/pypoetry/virtualenvs
ImportError while loading conftest '/home/mikehuang/Downloads/gnss_lib_py-1.0.2/tests/conftest.py'.
tests/conftest.py:14: in
from gnss_lib_py.navdata.navdata import NavData
gnss_lib_py/init.py:39: in
version = metadata.version("gnss-lib-py")
E importlib.metadata.PackageNotFoundError: No package metadata was found for gnss-lib-py
[mikehuang@localhost gnss_lib_py-1.0.2]$

System Information (please complete the following information):

  • OS: Fedora
  • OS Version: 39
  • Python Version: 3.11

Additional context

precise_ephemerides.py cannot be imported

Describe the bug
from gnss_lib_py.parsers.precise_ephemerides import parse_sp3 returns a ModuleNotFoundError

To Reproduce
Type the line of the bug description in a python console.
I use PyCharm and they suggest (through auto-completion) the 3 other files present in "gnss_lib_py/parsers", ie "ephemeris", "android" and "navdata", but not this "precise_ephemerides"

System Information (please complete the following information):

  • OS: Ubuntu 20.04 and Windows 10
  • Python Version: 3.10.8
  • gnss_lib_py: 0.1.7, installed using pip install gnss-lib-py

Plot Skyplot

1-plots the satellite skyplot using precise ephemerids sp3, it's possible?
2- How to read sp3 files.

Merge android-functionality

Ensure that all Android measurement functionality from deep_gnss and emd_fde is included in the repo code

Travis CI Bells and Whistles

  1. Add automatic tests for Python 3.6., 3.7. and 3.8.*
  2. (If possible) Add a line that automatically generates requirements.txt from pyproject.toml or vice-versa every time the code is pushed or built or whenever Travis CI does its thing

numpy.ufunc RuntimeWarning when building documentation

  • Ubuntu 18.04
  • python 3.8.9

I get the following errors when doing a clean build of the documentation:

Running Sphinx v4.1.0
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 20 source files that are out of date
updating environment: [new config] 20 added, 0 changed, 0 removed
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216, got 192
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216, got 192
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
  return f(*args, **kwds)
/usr/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: numpy.ufunc size changed, may indicate binary incompatibility. Expected 192 from C header, got 216 from PyObject
  return f(*args, **kwds)
reading sources... [100%] tutorial                                              
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [100%] tutorial                                               
generating indices... genindex py-modindex done
writing additional pages... search done
copying static files... done
copying extra files... done
dumping search index in English (code: en)... done
dumping object inventory... done
build succeeded.

The HTML pages are in build/html.

I initially thought it might be a numpy version mismatch between the requirements.txt file in docs/ and the poetry.lock file, but the issue still persisted even after updating the requirements.txt file.

Accepting pathlib paths in parser

Bug/Missing Functionality
Navdata parser requires CSV path to be a string, but for compatibility across operating systems and better functionality, it may be useful to allow pathlib.Path objects as well.

To Reproduce
pixel4_gt_path = pathlib.Path('/content/drive/path/to/train w derived/2021-04-29-US-SJC-2/Pixel4/ground_truth.csv')
gt_data_pixel4 = glp.AndroidGroundTruth2021(pixel4_gt_path)

See

def from_csv_path(self, csv_path, **kwargs):
"""Build attributes of NavData using csv file.
Parameters
----------
csv_path : string
Path to csv file containing data
header : string, int, or None
"infer" uses the first row as column names, setting to
None will add int names for the columns.
sep : char
Delimiter to use when reading in csv file.
"""
if not isinstance(csv_path, str):
raise TypeError("csv_path must be string")
if not os.path.exists(csv_path):
raise FileNotFoundError("file not found")
self._build_navdata()
pandas_df = pd.read_csv(csv_path, **kwargs)
self.from_pandas_df(pandas_df)

Specifically, line 105 only allows strings.

System Information (please complete the following information):

  • Run on Google Colab (June 26, 2023)
  • Python Version: Python 3.10.12

opcode 1I/Q/X is not supported for Beidou

Describe the bug
As explained by table 19 of https://files.igs.org/pub/data/format/rinex305.pdf, when reading a RINEX 3.02 file, both 1I/Q/X and 2I/Q/X observation codes should be accepted and treated the same as 2I/Q/X in the current RINEX standard. But gnss_lib_py doesn't support this. This will introduce following trace message

Traceback (most recent call last):
  File "/home/mikehuang/programs/./obs_featuring.py", line 20, in <module>
    staNavData = glp.RinexObs(file)
                 ^^^^^^^^^^^^^^^^^^
  File "/home/mikehuang/.local/lib/python3.11/site-packages/gnss_lib_py/parsers/rinex_obs.py", line 106, in __init__
    signal_type = signal_type_dict[constellation][band]
                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^
KeyError: '1I'

support multiple signal_type/observation_code in add_sv_states_rinex

Describe the proposed feature
My observation data has multiple signal and observation code, but add_sv_states_rinex prints 'Some ephemeris data is missing'. I have to add the sv states one by one

obs_signal=obs_data.where('gnss_id','gps').where('signal_type','l2').where('observation_code','2X')
sv_states=glp.add_sv_states_rinex(obs_signal)

Motivate the proposed feature
This features make it easier to use in case that multiple GNSS signals need to be considered.

rinex_nav: Incorrect time of clock

Bug Overview:
The clock correction model is being marked as updated for all satellites of a constellation when updated for any satellite from that constellation using rinex_nav.

Bug Description:
When a message with a new timestamp is parsed from a RINEX file, all satellites in the constellation that did not have a message at that time are updated with a 'dummy' record. The time of clock is being incorrectly set when dealing with these 'dummy' records such that all satellites are being treated as having an updated clock correction model when any satellite has an updated model.

data['t_oc'] = pd.to_numeric(data['time'] - consts.GPS_EPOCH_0.replace(tzinfo=None))
data['t_oc'] = 1e-9 * data['t_oc'] - consts.WEEKSEC * np.floor(1e-9 * data['t_oc'] / consts.WEEKSEC)

System Information:

  • OS: Windows 10
  • OS Version: 19045.3930
  • Python Version: 3.9.7

Verify function input/outputs have all been documented

Currently a few of the function parameter/returns are documented saying that we don't know exactly what the input/output size is. Or what a specific constant means, etc.

Verify that all functions input/outputs have been documented. This will help when we create tests in the future.

pandas SettingWithCopyWarning

Test all code and improve implementation to remove pandas SettingWithCopyWarning. Especially need to check android.py and ephemeris.py

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.