georust / rinex Goto Github PK
View Code? Open in Web Editor NEWRINEX and GNSS data processing :artificial_satellite:
License: Apache License 2.0
RINEX and GNSS data processing :artificial_satellite:
License: Apache License 2.0
I was getting an error with cargo test
on latest main:
error[E0432]: unresolved import `rinex::test_toolkit`
--> rinex/tests/compression.rs:4:9
|
4 | use rinex::test_toolkit;
| ^^^^^^^^^^^^^^^^^^^ no `test_toolkit` in the root
For more information about this error, try `rustc --explain E0432`
I had to add +#[cfg(feature = "tests")]
to rinex/tests/compression.rs
and rinex/tests/production.rs
to get it to run the tests.
Also the tests left two "untracked" git files rinex/merge.rnx
rinex/test.crx
. It would be nice to either add these to .gitignore
, or two write test output files to a temporary directory so not to touch the source tree.
I can't seem to find 3D IONEX maps anywhere (I mean complete maps, not truncated ones).
Theoretically, 3D IONEX is supported but we don't have any data to prove it does work correctly.
Also, that would be the good basis for several new CLI/Library features, like
Generate a meaningful library code coverage report with
cargo tarpaulin --workspace --all-features \
-t 300 -e rinex-cli -e crx2rnx -e rnx2crx -e ublox-rnx
--all-features
to cover the entire library--workspace -e $pkg
: exclude applications-t 300s
the default 60s limitation is easily exceeded by our testsCurrent code coverage is about 65% to this day.
Several cool features could be developed when provided Context is Meteo RINEX
There are two CVEs related to the old version of clap
we include, as seen in our audit.
If we simply update clap
then tests fail. But that is probably just because we need to change the test parameters.
Moreover, the YAML feature is deprecated and no longer documented, but not removed wholly. So after updating, we should probably look into moving to alternate ways of defining commands. Even though I prefer the YAML approach.
I'm thinking that being able to Hatanaka-compress output is something we'd like to have, since we can read it.
Version 0.9.2 doesn't compile properly anymore when trying to install via cargo in debian-slim (building the Dockerfile in https://github.com/larsnaesbye/test-rinextools) . I get 37 errors.
It would be nice to have CLK rinex files implement the preprocessing toolkit
Develop/add proper test benches for the following operations
Hi,
Running rinex-cli -m -h
produces confusing help;
How may I pass the input files to be merged?
Does it support many files to be merged together?
What is the correct order of -f -m -d
?
On rnx.gz Navigation files, how may I Filters out all IRNSS vehicles
or so? -I
does not work?
The example in wiki https://github.com/georust/rinex/wiki/file-merging
rinex-cli \
-f test_resources/CRNX/V3/ESBC00DNK_R_20201770000_01D_30S_MO.crx.gz \
-m test_resources/CRNX/V3/MOJN00DNK_R_20201770000_01D_30S_MO.crx.gz \
-o test.rnx.crx
[INFO rinex_cli] "WORKSPACE/ESBC00DNK_R_20201770000_01D_30S_MO/test.rnx.crx" has been generated
Does not work error: unexpected argument '-o' found
wherever I put -o
Now that this format is supported, integrate new features based on it
merge
operation
.sort()
generation
When at least one NAV file is provided, a skyplot view is feasible.
The current view is not working correctly
Use a standard library like geodesy::coord3d to replace the src/groundposition.rs
, so we can benefit from their methods
I failed to compile rinex 0.6.1 and got error[E0658]: use of unstable library feature 'vec_retain_mut'
.
I then updated Rust with rustup
from 1.59 to 1.63 and the error went away. A little research shows that the feature was marked stable as of 1.61.
So perhaps we should indicate in Cargo.toml
that the minimum Rust version is 1.61 so it can bail out early?
I took a brief look at https://docs.rs/hifitime/latest/hifitime/ and thought that maybe we can use it instead of the time
and chrono
crates?
Though SP3 isn't a *NEX file type it is pretty much a companion of the clock RINEX format and used by IGS and others to provide precise satellite orbits.
I can't see any other crate yet that supports SP3 so I think this crate would be a good choice to support it.
The format definition can be found on https://www.igs.org/formats-and-standards/. For some reason the "b" version is missing, can be found at https://files.igs.org/pub/data/format/sp3_glon.txt
Luckily its a fairly simple format type
The ANTEX record parsing does not properly handle all of the frequency bands available in current ANTEX files from IGS. I found this description the ANTEX format, and in the description of the START OF FREQUENCY
section it lists several potential frequency bands that could be encoded but the current parsing doesn't handle all of these cases. The current implementation appears to label anything it doesn't expect as either Carrier::L1
or Carrier::E1
. It also appears that all BeiDou frequency bands are given Galileo Carrier
values.
This behavior means it's impossible to differentiate which is the "real" L1 calibration value and which are the mislabeled values. Here is a minimal program, the output shows that the list of frequencies for at least some antennas are ambiguous since there are multiple entries with the same carrier value. I'm also attaching the ANTEX file that I'm using locally, but any relatively recent file from IGS demonstrates this issue.
fn main() {
let file_path = "/path/to/igs14.atx";
let rnx = rinex::Rinex::from_file(&file_path).unwrap();
if let rinex::record::Record::AntexRecord(atx) = rnx.record {
for (antenna, frequencies) in atx.iter() {
println!("Antenna {} {}", antenna.ant_type, antenna.sn);
for freq in frequencies.iter() {
println!("\tFrequency {:?}", freq.carrier);
}
}
}
}
skyplot
I've been experimenting with the position solving function of rinex-cli and have bumped into a problem when processing some Rinex files. In particular, I get the following panic from Rust:
thread 'main' panicked at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hifitime-3.9.0/src/epoch.rs:827:14:
invalid Gregorian date: Carry
stack backtrace:
0: rust_begin_unwind
1: core::panicking::panic_fmt
2: core::result::unwrap_failed
3: core::result::Result<T,E>::expect
at /usr/src/debug/rust/rustc-1.76.0-src/library/core/src/result.rs:1030:23
4: hifitime::epoch::Epoch::from_gregorian_utc
at ~/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hifitime-3.9.0/src/epoch.rs:826:9
5: rinex::epoch::parse_in_timescale
at ./rinex/src/epoch/mod.rs:228:25
6: rinex::epoch::parse_utc
at ./rinex/src/epoch/mod.rs:254:5
7: rinex::observation::record::is_new_epoch
at ./rinex/src/observation/record.rs:144:13
8: rinex::record::is_new_epoch
at ./rinex/src/record.rs:264:34
9: rinex::record::parse_record
at ./rinex/src/record.rs:396:29
10: rinex::Rinex::from_path
at ./rinex/src/lib.rs:811:34
11: rinex_cli::user_data_parsing
at ./rinex-cli/src/main.rs:110:28
12: rinex_cli::main
at ./rinex-cli/src/main.rs:156:24
13: core::ops::function::FnOnce::call_once
at /usr/src/debug/rust/rustc-1.76.0-src/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
./target/debug/rinex-cli -f tmp/mcso0820.24o -f tmp/mcso0820.24n -f tmp/igr23065.sp3 positioning --spp
I've noticed that the issue seems to be with the very end of the 24o file. If I trim a few records from the end then no crash occurs. I'm particularly suspicious of the line immediately before the splice. I don't know anything about the Rinex format, but that line looks different than anything else in the vicinity. If I remove that single line then no crash occurs. Here's the section in question:
19998277.560 106939857.756 6 1777.523 41.250 19998286.320
83175526.625 7 1382.520 43.500
24055982.500 128728438.271 6 3597.980 41.750 24055991.900
100122178.588 5 2798.426 36.750
4 1
RINEX FILE SPLICE; other post-header comments skipped COMMENT
24 3 22 3 0 0.0000000 0 20G05G10G13G15G16G18G23G26G27G29R03R04
R05R10R11R12R18R19R20R21
24800361.320 130327183.699 7 -3590.820 44.500 24800362.400
101553857.83747 -2798.039 43.250
22609837.440 118815681.988 8 2764.531 48.000 22609841.580
92583760.42549 2154.184 51.000
23145998.820 121633148.168 8 -1449.113 46.000 23146000.000
94779134.76645 -1129.180 36.750
And an except from a different 24o file with the same problem:
100267330.027 5 -2642.504 37.000
20836737.460 111462648.946 6 -2130.270 41.250 20836743.760
86693279.954 8 -1656.879 45.500
19998277.560 106939857.756 6 1777.523 41.250 19998286.320
83175526.625 7 1382.520 43.500
24055982.500 128728438.271 6 3597.980 41.750 24055991.900
100122178.588 5 2798.426 36.750
4 11
RINEX FILE SPLICE COMMENT
teqc 2019Feb25 NOAA/NOS/NGS/CORS 20240322 04:13:09UTCCOMMENT
Spider V7.8.1.9468 2024 03 22 04:00 COMMENT
BIT 2 OF LLI FLAGS DATA COLLECTED UNDER A/S CONDITION COMMENT
SNR is mapped to RINEX snr flag value [1-9] COMMENT
LX: >= 25dBHz -> 1; 26-27dBHz -> 2; 28-31dBHz -> 3 COMMENT
32-35dBHz -> 4; 36-38dBHz -> 5; 39-41dBHz -> 6 COMMENT
42-44dBHz -> 7; 45-48dBHz -> 8; >= 49dBHz -> 9 COMMENT
Product COMMENT
teqc.e edited: all SBAS satellites excluded COMMENT
teqc.e edited: all QZSS satellites excluded COMMENT
24 3 22 3 0 0.0000000 0 20G05G10G13G15G16G18G23G26G27G29R03R04
R05R10R11R12R18R19R20R21
24800361.320 130327183.699 7 -3590.820 44.500 24800362.400
101553857.83747 -2798.039 43.250
22609837.440 118815681.988 8 2764.531 48.000 22609841.580
92583760.42549 2154.184 51.000
23145998.820 121633148.168 8 -1449.113 46.000 23146000.000
94779134.76645 -1129.180 36.750
When formatting the CRINEX header, we emphasize this tool revision number.
As this number evolves we wind up not following the RINEX standards strictly.
This is also related to a testing method can be improved.
In https://github.com/gwbres/rinex/blob/main/rinex/db/NAV/navigation.json#L123-L126 it seems like the same key uraiNed
is present twice.
Replace old methods by having the preprocessor support LLI combinatorial conditions
Having the ability to load a pool of SP3 files could allow several calculations, like PPP (Precise Point Position solving).
It would be cool for the $WORKSPACE location to be possibly user defined.
Introduce a new command line argument like -w
--workspace
I have attempted to build the crate (both natively and in a Docker container) using both published versions and the git repo:
rinex = { git = "https://github.com/georust/rinex" }
rinex = { version = "0.16" }
In all cases, I'm seeing errors like the following
error[E0308]: `?` operator has incompatible types
--> /Users/USER/.cargo/registry/src/index.crates.io-6f17d22bba15001f/rinex-0.15.6/src/header.rs:773:56
|
773 | time_of_first_obs.time_scale = c
| ________________________________________________________^
774 | | .timescale()
775 | | .ok_or(ParsingError::TimescaleParsing(c.to_string()))?;
| |__________________________________________________________________________________^ expected `hifitime::TimeScale`, found `hifitime::timescale::TimeScale`
|
= note: `?` operator cannot convert from `hifitime::timescale::TimeScale` to `hifitime::TimeScale`
= note: `hifitime::timescale::TimeScale` and `hifitime::TimeScale` have similar names, but are actually distinct types
note: `hifitime::timescale::TimeScale` is defined in crate `hifitime`
--> /Users/USER/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hifitime-4.0.0-alpha/src/timescale/mod.rs:87:1
|
87 | pub enum TimeScale {
| ^^^^^^^^^^^^^^^^^^
note: `hifitime::TimeScale` is defined in crate `hifitime`
--> /Users/USER/.cargo/registry/src/index.crates.io-6f17d22bba15001f/hifitime-3.9.0/src/timescale.rs:70:1
|
70 | pub enum TimeScale {
| ^^^^^^^^^^^^^^^^^^
= note: perhaps two different versions of crate `hifitime` are being used?
It appears that gnss-rs is using hifitime v4.0.0-alpha
, while rinex itself is using v3.9.0
Hi, I might want to contribute to the development of rinex
, as my day to day work is in need of better tools than tecq, Anubis, gfzrnx etc. I have experience with RINEX, and though fairly new to Rust, I'm already fascinated by the tooling and community.
Are there some smaller tasks you'd like to be worked on first, like contributing test data, hardening or fixing the epochs?
In the orbits.json
file there are a few typos I spotted:
Under the GPS RINEX 4 section (actually seems to be for CNAV):
deltaNoDot
-> deltaNDot
uraiNed
-> uraiEd
(note this is different to the uraiNed0
/uraiNed1
/uraiNed2
fields which are correctiscL1C
-> iscL2C
iscL1Cd
and iscL1cCp
should be deleted - don't exist for CNAV, but do for CNV2 (so add back when #78 implemented)I ended up having to modify the orbits.json
file as I was having problems with the decoding of the health bits. I think this was mainly due to the mix of LNAV and CNAV types in there at the moment. The different ephemeris types tend to have quite different definitions of how they encode the health information.
IRNSS (NavIC) the indian Constellation is not tested enough
The hifitime Epoch
struct supports different timescales but the decoder often initialises them with the wrong types.
For reference here is how each type of RINEX file should set the timescale:
TIME OF FIRST OBS
header lineG01
use GPS timescale, E01
use Galileo etcWhen upgrading to newer (>0.18) versions of strum
/strum-macros
compilation fails, probably because 0.19+ no longer is silent about errors.
error[E0599]: no variant or associated item named `from_str` found for enum `Observable` in the current scope
--> rinex/src/header.rs:466:72
|
466 | if let Ok(observable) = meteo::observable::Observable::from_str(phys_str.trim()) {
| ^^^^^^^^ variant or associated item not found in `Observable`
|
::: rinex/src/meteo/observable.rs:10:1
|
10 | pub enum Observable {
| ------------------- variant or associated item `from_str` not found here
|
= help: items from traits can only be used if the trait is implemented and in scope
= note: the following trait defines an item `from_str`, perhaps you need to implement it:
candidate #1: `FromStr`
Is this perhaps because EnumString no longer implements the same function?
I have some time to read through the new orbits.json
file, and I am comparing it to the modified one I was using from a few months ago and making notes of anything I spot.
GPS CNV2:
QZSS CNV2:
BDS CNV1:
"spare"
- VSCode was highlighting this, should use spare1
, spare2
t_op
:b1c
- it might be clearer that this field has special meaning by adding integrity
, or if
to the nameBDS CNV2:
"spare"
- VSCode was highlighting this, should use spare1
, spare2
t_op
:isc_b1cd
with a spare, then add ISC B2ad between the spare and TGDs:b1c
to b2a_b1c
, possibly adding integrity to name like suggestion above:BDS CNV3:
"spare"
- VSCode was highlighting this, should use spare1
, spare2
t_op
:isc_
and tgd_
values - should go from sisa1_oc2
to sismai
:b1c
to b2b
, possibly adding integrity to name like suggestion aboveiodc
with tgd_b2bi
iode
To help with the test coverage I have attached the IGS RINEX v4 "BRDC" file I was using earlier in the year. This is a useful file as it has every type of NAV data except for GPS CNV2, and BDS CNV3.
BRD400DLR_S_20230710000_01D_MN.rnx.gz
I will see if there is the missing ones in any newer files.
I'm looking for "high precision" Observation data, where data are scaled by a fixed scalar.
A scaling is used in advanced contexts to improve the precision over the data, and in close relation with the receiver capacity.
Having such a file would allow proper testing of such advanced usage, demonstrate we can deal with it correctly and test data scaling related methods.
@larsnaesbye, do you think SFDI would have such data ?
I noticed that some repos have automatically built binaries upon release. In our case I guess this would be the rinex-cli
, crx2rnx
, and rnx2crx
binaries.
This Github Action might be able to do this. People can then check out the executables without having to install the full Rust toolset.
So called "qc" Quality Check - mainly statistical analysis, historically introduced by the teqc
command line tool
When loading current data from NASA (which uses RINEX version 2) it panics when it has a blank constellation. Using data from NASA requires a username and password, but the link is: https://cddis.nasa.gov/archive/gnss/data/daily/2024/brdc/brdc0990.24n.gz
The header from the RINEX we see is
2 NAVIGATION DATA RINEX VERSION / TYPE
CCRINEXN V1.6.0 UX CDDIS 08-APR-24 17:18 PGM / RUN BY / DATE
IGS BROADCAST EPHEMERIS FILE COMMENT
0.2142D-07 0.1490D-07 -0.1192D-06 -0.5960D-07 ION ALPHA
0.1208D+06 0.3277D+05 -0.2621D+06 0.1311D+06 ION BETA
0.931322574615D-09 0.355271367880D-14 319488 2309 DELTA-UTC: A0,A1,T,W
18 LEAP SECONDS
END OF HEADER
1 24 4 8 0 0 0.0 0.178544782102D-03 0.238742359215D-11 0.000000000000D+00
0.450000000000D+02-0.582500000000D+02 0.359050670187D-08-0.282724798209D+01
The RINEX Version 2 requirement is that it support "blank" constellation, but none of the test files used in your repo have this.
10. RINEX VERSION 2 FORMAT DEFINITIONS AND EXAMPLES
+----------------------------------------------------------------------------+
| TABLE A1 |
| OBSERVATION DATA FILE - HEADER SECTION DESCRIPTION |
+--------------------+------------------------------------------+------------+
| HEADER LABEL | DESCRIPTION | FORMAT |
| (Columns 61-80) | | |
+--------------------+------------------------------------------+------------+
|RINEX VERSION / TYPE| - Format version (2) | I6,14X, |
| | - File type ('O' for Observation Data) | A1,19X, |
| | - Satellite System: blank or 'G': GPS | A1,19X |
| | 'R': GLONASS | |
| | 'T': NNSS Transit | |
| | 'M': Mixed | |
This just looks like an oversight and should be an easy fix.
I tried with 4.0.9, and the feature has been removed silently. So if we ever need the clap 4 features then we have to move to the alternative ways of specifying commands. Right now, I don't think it's a problem, though.
Some info can be found here.
Use something like
uses: actions-rs/toolchain@v1
with:
toolchain: 1.xx.y
override: true
components: rustfmt, clippy
processing
toolkit must be tested more thoroughlyWith chrono 0.4.23 some functions are deprecated: chronotope/chrono#827
Compiling sinex v0.1.0 (/Users/lanch/Desktop/Dev/rinex/sinex)
warning: use of deprecated associated function `chrono::NaiveDate::and_hms`: use `and_hms_opt()` instead
--> sinex/src/datetime.rs:20:11
|
20 | Ok(dt.and_hms(h as u32, m as u32, s as u32))
| ^^^^^^^
|
= note: `#[warn(deprecated)]` on by default
In RINEX v2 files header records with labels ION ALPHA
and ION BETA
are allowed. They contain coefficients for ionospheric delay calculations for GPS system (GLONASS, which is also allowed in those RINEX versions, does not broadcast any ionosphere-related information and can be safely ignored). Those fields are checked for by header parser but not handled any further (there are TODO notices).
According to Navipedia and IS-GPS-200D (Figure 20-4), coefficients are used for Klobuchar model calculations. This model is already supported by library. IONOSPHERIC CORR
field, which replaced ION ALPHA
and ION BETA
in RINEX v3, is handled by header parser which presents it as a separate IonMessage
. I propose that ION ALPHA
and ION BETA
should be handled in a similar manner, adding a IonMessage
wrapping KbModel
to resulting Header
.
I am willing to make a PR for this issue. It would be my first experience with Rust though, possibly requiring thorough code review (I have previous coding experience in other languages, so maybe it's not that bad). Current developers/maintainers, would you like me to make such a PR?
Originally posted by ljbade March 17, 2023
With RINEX 4 it is now possible to get broadcast ephemerides for the newer signals such as GPS CNAV or BDS CNAV. However it seems the current implementation only recognises the ephemeris types from RINEX 3 such as LNAV.
I am interested in computing the position difference between the different ephemeris types for the same satellite so it would be useful to add support for:
The recent IGS broadcast navigation files have examples of most of these new formats. An example is https://cddis.nasa.gov/archive/gnss/data/daily/2023/brdc/BRD400DLR_S_20230750000_01D_MN.rnx.gz
The IGS file doesn't seem to yet have an example of GPS CNAV-2 yet, which is from the GPS L1C signal.
Current implementation is fine performance wise, thanks to correct BufferedReader usage,
but it could use a little more abstraction as similarly done in the georust GPX parser
This would allow parsing from pushing into any abstract streaming interface
I tried not supplying any parameters to see if I got some help, but instead it panics:
lars@machine debug % ./rinex-cli
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', rinex-cli/src/main.rs:432:31
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
So I guess it should either abort with an error message or default to displaying a help - which the clap
function Command::arg_required_else_help
(2.x) or arg_required_else_help
(3.x) can do :-)
With newer (<0.18) versions of strum, compilation of rinex
fails. One of the things mentioned in 0.19 is that "Most of the strum plugins will now error more aggresively on invalid options being used. Historically, the plugins have ignore invalid options, but most of these should error now.".
One of the reported errors is that deserialize
is unknown:
error: expected one of: `message`, `detailed_message`, `serialize`, `to_string`, `disabled`, `default`, `ascii_case_insensitive`, `props`
--> rinex/src/navigation/record.rs:30:32
|
30 | #[strum(serialize = "EPH", deserialize = "EPH")]
| ^^^^^^^^^^^
Indeed, if I try to rename deserialize
to anything else under 0.18, it makes no difference. And in newer versions it is not mentioned in the documentation.
The question is if serialize
is enough to handle deserialization as well. The documentation isn't very clear on this...?
A new doris
branch attempts at answering Discussion #77
The fundemental issue is how to correctly describe the DORIS measurements in time.
They are published in the DORIS timescale, with means to relate the observations to TAI.
My plan, since we cannot represent "DORIS", is to find means to store them as TAI by applying the correct offset.
Not sure this is entirely feasible at the moment.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.