iannesbitt / readgssi

python tool to read and plot Geophysical Survey Systems Incorporated (GSSI) radar data

• 😄 Pronouns: he/him
• 🧑‍💻 Website: iannesbitt.org
• 🪪 ORCiD: 0000-0001-5828-6070
• ⛷️ Enjoys: nordic skiing, alpine skiing, biking, futbol, football, golf, tea, coffee, pesto, macaroni and cheese
• 💼 Current employment: Research Software Engineer, National Center for Ecological Analysis and Synthesis, UCSB
• 🔭 Working on a Master's project investigating sediment delivery to freshwater lakes in Maine since deglaciation
• 📜 Previous employment:
  • 🇦🇶 Antarctic field expedition I-165-M to Allan Hills Blue Ice Area (field season 2019-2020, geophysical survey lead)
  • 🌋 Raspberry Shake (2 yrs, tech support/software development/science communication)
  • ⛴️ e4sciences LLC (3.5 yrs, marine/terrestrial geophysical survey engineer)
• 📰 Published work: Google Scholar, ORCiD, ResearchGate
• 📄 Resume: download
• 📄📄 CV: download
• ⚡ Fun fact: there are only two ATMs in Antarctica

readgssi's Issues

Another KeyError Issue -- GSSI StructureScan Mini

Hey! Having some KeyError issues with my GSSI StructureScan Mini DZT files. It seems to be the same type of errors from #11 and #7. I tried installing from source and it didn't help.

Here are my DZT files if you need them:
SSMini_DZT_Files.zip

Excited to continue playing with this tool. Appreciate your work!

Thanks!

Unable to read DZG files

When trying to open any GPR transect (DZT file) that has associated GPS info (DZG file), I am unable to read the DZG file info:

$ readgssi -i MI_MANISTEE__021.DZT -p 5
Traceback (most recent call last):
  File "c:\programdata\anaconda3\envs\readgssi\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\programdata\anaconda3\envs\readgssi\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\ProgramData\Anaconda3\envs\readgssi\Scripts\readgssi.exe\__main__.py", line 7, in <module>
  File "c:\programdata\anaconda3\envs\readgssi\lib\site-packages\readgssi\readgssi.py", line 489, in main
    showmarks=showmarks)
  File "c:\programdata\anaconda3\envs\readgssi\lib\site-packages\readgssi\readgssi.py", line 89, in readgssi
    r = readdzt(infile, gps=normalize, spm=spm, start_scan=start_scan, num_scans=num_scans, epsr=epsr, antfreq=antfreq, verbose=verbose)
  File "c:\programdata\anaconda3\envs\readgssi\lib\site-packages\readgssi\dzt.py", line 223, in readdzt
    gps = readdzg(infile_gps, 'dzg', header, verbose=verbose)
  File "c:\programdata\anaconda3\envs\readgssi\lib\site-packages\readgssi\gps.py", line 147, in readdzg
    z0 = array['altitude'].iat[rowgga]
  File "c:\programdata\anaconda3\envs\readgssi\lib\site-packages\pandas\core\indexing.py", line 2025, in __getitem__
    return self.obj._get_value(*key, takeable=self._takeable)
  File "c:\programdata\anaconda3\envs\readgssi\lib\site-packages\pandas\core\series.py", line 986, in _get_value
    return self._values[label]
IndexError: index 11 is out of bounds for axis 0 with size 10

If I copy the DZT file to a new directory and open that then I don't get the out of bounds error. Assuming this is an issue with the reading the DZG file strings.

'sampling depth' is ambiguous

rhf_depth is not sampling depth. Sampling depth is actually rhf_depth - rhf_top.

"Greys" colormap flips polarity

BWB is rendered as WBW! How could I not have noticed this before...

Switching to "gray" colormap, which renders polarity properly, provides extra contrast, and shifts the mid-level brightness down...all probably good things.

documentation?

Is documentation for this project in the works? I see there is a Python package - could you all at lease autodoc the API on ReadtheDocs?

How to get time_range for selected DZT File

Hello,
I'm trying to understand this package, so if you want to read a DZT file you need to type this commande :
hdr, arrs, gps = readgssi.readgssi(infile='COMMON__030.DZT', verbose=True)
after that you get a couple of informations like :

2021-04-23 10:45:41 - traces per second:  110.0
2021-04-23 10:45:41 - traces per meter:   300.0
2021-04-23 10:45:41 - epsr:               1.0
2021-04-23 10:45:41 - speed of wave:      3.00E+08 m/sec (99.99% of vacuum)
2021-04-23 10:45:41 - time range (TWTT):  10.0 ns
2021-04-23 10:45:41 - sampling depth:     1.7 m
2021-04-23 10:45:41 - "rhf_top":          -0.2 m
2021-04-23 10:45:41 - "rhf_depth":        1.5 m
2021-04-23 10:45:41 - offset to data:     1024 bytes
2021-04-23 10:45:41 - traces:             3608
2021-04-23 10:45:41 - seconds:            32.80000000
2021-04-23 10:45:41 - array dimensions:   256 x 3608

my question is how i can acces to this informations, for exemple i want to get time range as a variable and use it in same plotting

Thank you

'1.5/1.6GHz' antenna

from Rahul T:

I'm trying to use your library and I am facing difficulty with a simple thing. The DZT file I have was scanned using frequency 1.5 GHz and I am wondering if there's a way to use the frequency while reading the DZT file.

2021-08-04 12:37:57 - WARNING: could not read frequency for antenna name "'1.5/1.6GHz'"
2021-08-04 12:37:57 - WARNING: trying to use frequencies of 1 MHz (estimated)...
2021-08-04 12:37:57 - more info: rh_ant=['1.5/1.6GHz', None, None, None]
2021-08-04 12:37:57 -            known_ant=[False, None, None, None]

.DZT Example and help

Hello Ian. I'm working with some vintage seismic data which was scanned very poorly and save as tiff images. I was advised to look into treating them as gpr data to analyze the depths but have not been able to convert them into .dzt format. My plan was to work through your examples but I see the .dzt file are not available. Is it possible that you give me access to them? I would also like to know if I can use one of the files to alter the header and use my numpy data to get it to display.

Your help in this will be greatly appreciated.

My email address is [email protected]

compare with Radan 7, got different results

i use readgssi process this dzt file, compare with radan 7, but got different results, where did i set wrong? thank u!

GeoPy throws error when processing when GPS Altitude changes

@iannesbitt , first of all, thank you for the effort on putting this library together. It is a very handy tool!

However, I encountered an issue trying to use it. I am currently trying to use the main function readgssi.readgssi to open a dataset that contains DZT and DZG files. The DZG files contain the information from the GPS including altitude. Therefore, when the main function calls the function readdzg, GeoPy throws the following error: "ValueError: Calculating distance between points with different altitudes is not supported". This seems to be documented under GeoPy - Calculate Distance
When I open the data without the GPS the function works perfectly.

HERE a sample of the data I am using. Please, let me know if you need more info or a extra hand as contributor.

Interoperability with PyVista

This project looks really promising! Next time I'm working on a project with GPR data, I'm going to have to use this!

I saw that this project reads data from the DZT format - correct me if I'm wrong, but those files have XYZ trace locations in the header, right? If this project can read all that data, perhaps we could collaborate on making an interface between this softaware and vista: https://docs.pyvista.org for 3D rendering of many GPR lines.

I've done this before and I am envisioning a way to make figures like these with ease:

Main Function Documentation

Not sure if it says this anywhere in the documentation but the output of the main readgssi.readgssi() function claims to be a numpy array but it is actually a dictonary of numpy arrays with the keys being the channel numbers.

pandas frame.append method has been removed

Hi, firstly - thanks for making such a great program!

I was getting the error AttributeError: 'DataFrame' object has no attribute 'append' when running the line
readgssi.readgssi(infile='PROJECT_001.DZT', frmt=None, verbose=True)

and when searching for what might be the problem came across this stackoverflow answer saying that since Pandas 2.0, the frame.append method has been removed.

I've fixed my situation temporarily by downgrading to Pandas 1.5.3 and just wanted to let you know. If I get any time I'll make a pull request with the append method changed to the recommended concat method.

two-way travel time is calculated incorrectly

Previously, plots were erroneously drawn with one-way travel time rather than two-way travel time, while the label said "Two way time". This is fixed in commit e47abd6 (version 0.0.15)

Question: Hiding header output

Is there any way to hide the header information output in python?

Multi-frequency antenna, header problem

Hello, there seems to be a problem when reading multi channel *.DZT files from a GSSI SIR4000 console, with a multi frequency antenna 300MHz/800MHz. It looks like in their header file they do not separate properly the info related to the different channels. It is already somehow buggy in the way it looks on RADAN.

From the command line, I use:

readgssi -N -f gprpy -i CAV_2Z2-01__001.DZT

(input file attached as a ZIP file
CAV_2Z2-01__001.zip
)

Hereinafter the WARNING messages and the error:

...
--------------------WARNING - PLEASE READ---------------------
2021-02-08 08:10:30 - WARNING: could not read frequency for antenna name "'300/800D'"
2021-02-08 08:10:30 - WARNING: trying to use frequencies of None MHz (estimated)...
2021-02-08 08:10:30 - more info: rh_ant=['300/800D', 'none', None, None]
2021-02-08 08:10:30 -            known_ant=[False, False, None, None]
2021-02-08 08:10:30 - please submit a bug report with this warning, the antenna name and frequency
2021-02-08 08:10:30 - at https://github.com/iannesbitt/readgssi/issues/new
2021-02-08 08:10:30 - or send via email to ian (dot) nesbitt (at) gmail (dot) com.
2021-02-08 08:10:30 - if possible, please attach a ZIP file with the offending DZT inside.
--------------------------------------------------------------
--------------------WARNING - PLEASE READ---------------------
2021-02-08 08:10:30 - WARNING: could not read frequency for antenna name "'none'"
2021-02-08 08:10:30 - WARNING: trying to use frequencies of None MHz (estimated)...
2021-02-08 08:10:30 - more info: rh_ant=['300/800D', 'none', None, None]
2021-02-08 08:10:30 -            known_ant=[False, False, None, None]
2021-02-08 08:10:30 - please submit a bug report with this warning, the antenna name and frequency
2021-02-08 08:10:30 - at https://github.com/iannesbitt/readgssi/issues/new
2021-02-08 08:10:30 - or send via email to ian (dot) nesbitt (at) gmail (dot) com.
2021-02-08 08:10:30 - if possible, please attach a ZIP file with the offending DZT inside.
--------------------------------------------------------------
2021-02-08 08:10:30 - beginning processing for channel 0 (antenna 300/800D)
2021-02-08 08:10:30 - no gps information for distance normalization
Traceback (most recent call last):
  File "/home/alex/.virtualenvs/gpr/bin/readgssi", line 8, in <module>
    sys.exit(main())
  File "/home/alex/.virtualenvs/gpr/lib/python3.8/site-packages/readgssi/readgssi.py", line 480, in main
    readgssi(infile=infile, outfile=outfile, antfreq=antfreq, frmt=frmt, plotting=plotting, dpi=dpi,
  File "/home/alex/.virtualenvs/gpr/lib/python3.8/site-packages/readgssi/readgssi.py", line 146, in readgssi
    header, data[ar], gps = arrayops.distance_normalize(header=header, ar=data[ar], gps=gps,
  File "/home/alex/.virtualenvs/gpr/lib/python3.8/site-packages/readgssi/arrayops.py", line 162, in distance_normalize
    return header, proc, gps
UnboundLocalError: local variable 'proc' referenced before assignment

Sorry for bothering you with an issue which is probably related to a GSSI bug, and thank you again for your efforts!
Best,
Alessandro

Errors opening files or help (from GitBash)

When trying to run the help or open a DZT file I keep getting the following error:

$ readgssi -h
Traceback (most recent call last):
  File "c:\programdata\anaconda3\envs\readgssi\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\programdata\anaconda3\envs\readgssi\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\ProgramData\Anaconda3\envs\readgssi\Scripts\readgssi.exe\__main__.py", line 7, in <module>
  File "c:\programdata\anaconda3\envs\readgssi\lib\site-packages\readgssi\readgssi.py", line 275, in main
    fx.printmsg(config.help_text)
  File "c:\programdata\anaconda3\envs\readgssi\lib\site-packages\readgssi\functions.py", line 12, in printmsg
    print('%s - %s' % (datetime.now().strftime('%Y-%m-%d %H:%M:%S'), msg))
  File "c:\programdata\anaconda3\envs\readgssi\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f12f' in position 134: character maps to <undefined>

KeyError for GSSI files

As with #7, I'm running into a KeyError when loading a true GSSI .DZT file thus one not simulated by gprMax:

Would it be possible to include this header as well?

Thank you :)

Crash report - probable antenna frequency issue

Error below:

2020-04-01 21:59:05 - readgssi 0.0.16
2020-04-01 21:59:05 - reading...
2020-04-01 21:59:05 - input file:         RADARDATA/PROJECT1.PRJ/FILE__001.DZT
Traceback (most recent call last):
  File "/home/gssi/anaconda3/envs/readgssi/lib/python3.7/site-packages/readgssi/dzt.py", line 133, in readdzt
    header['antfreq'][chan] = ANT[header['rh_antname'][chan]]
KeyError: 'none'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/gssi/anaconda3/envs/readgssi/bin/readgssi", line 8, in <module>
    sys.exit(main())
  File "/home/gssi/anaconda3/envs/readgssi/lib/python3.7/site-packages/readgssi/readgssi.py", line 462, in main
    spm=spm, epsr=epsr, title=title, zoom=zoom)
  File "/home/gssi/anaconda3/envs/readgssi/lib/python3.7/site-packages/readgssi/readgssi.py", line 79, in readgssi
    r = readdzt(infile, gps=normalize, spm=spm, epsr=epsr, verbose=verbose)
  File "/home/gssi/anaconda3/envs/readgssi/lib/python3.7/site-packages/readgssi/dzt.py", line 137, in readdzt
    header['antfreq'][chan] = int("".join(takewhile(str.isdigit, header['rh_ant'][chan].replace('D5','').replace('D6','')))) # hoping this works
ValueError: invalid literal for int() with base 10: ''

DZT_DZX 1-3.zip

Exporting GPR Data of SIR-30 Multi-Channel System to A CSV/TXT File

Dear Mr. Ian Nesbitt and Your Team,

I currently use the GSSI SIR-30 multi-channel GPR system. Is it possible to export GPR data arrays for all channels (for example, in a CSV or TXT format)? If not, how can I read the GPR data for each single channel? It seems that, by default, the readgssi program only read/plot GPR data on the first channel.

I would appreciate your response.

Thank you very much!

readgssi not reading binary csv radargram file

I converted a dzt file into a csv and ran a PCA on the file, but I wasn't able to reread the edited csv into readgssi.

2024-01-16 23:38:04 - reading...
2024-01-16 23:38:04 - input file:         /content/drive/MyDrive/Link portal/rgssi/09_13_2023_perroot/1600MHz/Processed_Outputs/dzt2csv/5cm_spacing/001.csv
---------------------------------------------------------------------------
IndexError                                Traceback (most recent call last)
<ipython-input-5-3dd721c2cf13> in <cell line: 1>()
----> 1 readgssi.readgssi(infile=r'/content/drive/MyDrive/Link portal/rgssi/09_13_2023_perroot/1600MHz/Processed_Outputs/dzt2csv/5cm_spacing/001.csv', frmt=None, verbose=True, histogram=False,)

1 frames
/usr/local/lib/python3.10/dist-packages/readgssi/dzt.py in readdzt(infile, gps, spm, start_scan, num_scans, epsr, antfreq, verbose, zero)
    204         else:
    205             infile.seek(98 + (MINHEADSIZE*(chan))) # start of antenna bytes for channel n
--> 206         header['dzt_ant'][chan] = infile.read(14)
    207         header['rh_ant'][chan] = header['dzt_ant'][chan].decode('utf-8').split('\x00')[0]
    208         header['rh_antname'][chan] = header['rh_ant'][chan].rsplit('x')[0]

IndexError: list assignment index out of range

CSV Export

I used the write to csv function and I had a couple of questions.

I was curious what the raw values in the exported csv in fact are. Are they 32 bit signed integers? From the header that gets created, it appears that they are.

I was also curious if zero represents zero volts with GSSI, would the conversion be a linear one? Or are these power values? It seems to be nowhere in the GSSI documentation about what format these values actually get stored in.

Thank you!

fix for #15 causes mishandling of distance normalization

These lines, put in place to address #15, cause GPS information to be incorrectly applied across the array.

readgssi/readgssi/arrayops.py

Lines 99 to 102 in b0b43c0

	c = np.repeat(c, norm_vel['normalized'].astype(int, casting='unsafe').values[on:on+c.shape[1]], axis=1)
	c = reducex(c, by=nvm, chnum=i, number=nvm, verbose=verbose)
	proc = np.concatenate((proc, c), axis=1)
	on += c.shape[1]

Additionally, the proc array never gets incorporated into the returned array.

readgssi/readgssi/arrayops.py

Line 108 in b0b43c0

return header, ar, gps

Reduce memory usage in distance normalization

During distance normalization, radar array is expanded all at once, which is untenable for medium (0.1 GB) to large files because the expanded array can exceed the RAM size (speaking from experience). Lines 96 and 99 of arrayops.py (below) are the culprits and should be modified.

readgssi/readgssi/arrayops.py

Lines 96 to 99 in bb3bb2c

	ar = np.repeat(ar, norm_vel['normalized'].astype(int, casting='unsafe').values, axis=1)
	nvm = int(round(norm_vel['normalized'].mean()))
	del norm_vel
	ar = reducex(ar, by=nvm, verbose=verbose)

Solution: chunk-ify distance normalization by breaking the original array into int(norm_vel['normalized'].mean()) + 1 parts and tacking each normalized chunk on to a processed array to return. That way, max memory usage should only be ~3x filesize, instead of norm_vel['normalized'].mean() *filesize (which is far too often something extreme like 75x).

This will require a slicing for loop around the np.repeat() and readgssi.arrayops.reducex() functions to build the new array block by block.

header should be read in entirety for each channel

header is replicated for each channel, meaning that each channel should have entirely unique header values.

SIR-3000 antenna names are different than those generated by the SIR-4000

readgssi thinks samples per meter is negative in this command?

For some reason, this command yields a "must be positive" error like below, but ends up processing the file just fine.

$ readgssi -p 4 -x cm -z cm -g 3 -d 161 -i ./File____009.DZT 
2019-06-20 22:32:51 - ERROR: samples per meter must be positive
2019-06-20 22:32:51 - readgssi 0.0.12
2019-06-20 22:32:51 - reading...
2019-06-20 22:32:51 - input file:         ./File____009.DZT
2019-06-20 22:32:51 - success. header values:
2019-06-20 22:32:51 - system:             StructureScan Mini (system code 7)
2019-06-20 22:32:51 - antennas:           ['SS MINI', None, None, None]
2019-06-20 22:32:51 - ant 0 frequency:    1600 MHz
2019-06-20 22:32:51 - date created:       2017-04-08 08:40:22+00:00
2019-06-20 22:32:51 - date modified:      (never modified)
2019-06-20 22:32:51 - gps-enabled file:   no
2019-06-20 22:32:51 - number of channels: 1
2019-06-20 22:32:51 - samples per trace:  256
2019-06-20 22:32:51 - bits per sample:    32 signed
2019-06-20 22:32:51 - traces per second:  300.0
2019-06-20 22:32:51 - traces per meter:   161.00 (manually set - value from DZT: 393.70)
2019-06-20 22:32:51 - epsr:               8.0
2019-06-20 22:32:51 - speed of light:     1.06E+08 m/sec (35.35% of vacuum)
2019-06-20 22:32:51 - sampling depth:     0.4 m
2019-06-20 22:32:51 - "rhf_top":          0.0 m
2019-06-20 22:32:51 - offset to data:     1024 bytes
2019-06-20 22:32:51 - traces:             1131
2019-06-20 22:32:51 - seconds:            3.77000000
2019-06-20 22:32:51 - array dimensions:   256 x 1131
2019-06-20 22:32:51 - image stats
2019-06-20 22:32:51 - mean:               -422770.8228959438
2019-06-20 22:32:51 - stdev:              2769389.6223992216
2019-06-20 22:32:51 - lower color limit:  -8730939.690093609 [mean - (3 * stdev)]
2019-06-20 22:32:51 - upper color limit:  7885398.04430172 [mean + (3 * stdev)]
2019-06-20 22:32:51 - xmax: 702.4844720496894 Distance (cm), zmax: 40.63999950885773 Depth at $\epsilon_r$=8.0 (cm)
2019-06-20 22:32:51 - attempting to plot with colormap Greys
2019-06-20 22:32:52 - saving figure as ./File____009c0G3.png
2019-06-20 22:32:52 - showing matplotlib figure...

Issue plotting the file

Hi Ian,

many thanks for sharing this tool. It is very useful for many of us using gprRadars.

I have one issue and to be honest, I wonder that I'm the first one to point that.
I followed the documentation step by step but when I want to plot the data I get the following error:

>>> from readgssi import readgssi
>>> readgssi.readgssi( infile='metal_pipe_merged.dzt', outfile='09.png', frmt=None, zero=0, plot=6 )
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
TypeError: readgssi() got an unexpected keyword argument 'plot'

I installed readgssi today. The version is:
readgssi 0.0.16

If I remove the plot argument it works.
Using the BASH command line it plots the radargramm as expected.

Can you solve this issue?

Problem reading a DZT file

Hi,

I try to read a .dzt file from a Structure Scan Mini device from GSSI but I get the following error:

>>> readgssi.readgssi( infile='File001.DZT', outfile='0a.png', frmt=None, zero=0, plotting=True )
Traceback (most recent call last):
  File "/home/wilhem/anaconda3/envs/readgssi/lib/python3.7/site-packages/readgssi/dzt.py", line 133, in readdzt
    header['antfreq'][chan] = ANT[header['rh_antname'][chan]]
KeyError: 'SS MINI #338\n'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/wilhem/anaconda3/envs/readgssi/lib/python3.7/site-packages/readgssi/readgssi.py", line 79, in readgssi
    r = readdzt(infile, gps=normalize, spm=spm, epsr=epsr, verbose=verbose)
  File "/home/wilhem/anaconda3/envs/readgssi/lib/python3.7/site-packages/readgssi/dzt.py", line 137, in readdzt
    header['antfreq'][chan] = int("".join(takewhile(str.isdigit, header['rh_ant'][chan].replace('D5','').replace('D6','')))) # hoping this works
ValueError: invalid literal for int() with base 10: ''

It seems to me that the program didn't recognize the device

    header['antfreq'][chan] = ANT[header['rh_antname'][chan]]
KeyError: 'SS MINI #338\n'

but I can be wrong.

I cannot upload the file here but I can send it directly to you, if I have your email address

Errors reading GGA strings other than $GPGGA

Bug caught by examining example DZG from @fxsimon which uses $GNGGA strings instead of $GPGGA.

Solution: make search for line strings containing 'GGA' broader.

Plots of short lines are tall and thin, and have extra white space

Users may notice that when plotting short lines (a few meters' length) the automatic sizing gets thrown off, and plots are rendered tall and thin, with lots of white space on the top and bottom. See example below. This is a known bug and will be fixed in the next few months. Until then, plots can be resized in the Matplotlib GUI. Sorry for the inconvenience and thanks for your patience.

Affected plots look like this:

They should look like this:

Handling of line pause during collection with GPS input on SIR-4000

Currently (in both RADAN and readgssi) pauses are handled incorrectly. The issue seems to stem primarily from the fact that GPS data collection continues while trace collection is paused. This results in improperly aligned trace numbers between GSSI proprietary NMEA sentences and radar array scan columns, and thus incorrect post-pause pick export locations (in RADAN) and an area of extreme horizontal shortening due to improper distance normalization (in readgssi).

Possible solutions include:

1. (for readgssi only) auto identification of pauses, based on one or more of the following:
   • system mark location(s) in radar array
   • large misalignment of scan numbers at line end
   • large change in survey speed, followed by resume

• this would be followed by a scan number shift in readgssi's GPS array

2. (for both readgssi and RADAN) NMEA scan reassignment
   • a utility that identifies pauses and reassigns scan numbers to NMEA sentences in .DZG files

• this would require knowing/verifying the end scan number of the radar array

edit: option 2 was implemented

GPS reading, datetime error

Hey there. I am trying to use certain modules within python to read GPS data.

I'm getting the following error :

dzt.readdzt("TUK_MESSUNGEN_KATALOG_002.DZT")
2023-05-15 16:18:12 - WARNING: no time zero specified for channel 0, defaulting to rh_zero value (1)
2023-05-15 16:18:12 - WARNING: no time zero specified for channel 1, defaulting to rh_zero value (1)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\Riedel\.conda\envs\readgssi\lib\site-packages\readgssi\dzt.py", line 306, in readdzt
    gps = readdzg(infile_gps, 'dzg', header, verbose=verbose)
  File "C:\Users\Riedel\.conda\envs\readgssi\lib\site-packages\readgssi\gps.py", line 97, in readdzg
    ts0 = TZ.localize(datetime.combine(msg.datestamp, msg.timestamp)) # row 0's timestamp (not ideal)
  File "C:\Users\Riedel\.conda\envs\readgssi\lib\site-packages\pytz\__init__.py", line 245, in localize
    raise ValueError('Not naive datetime (tzinfo is already set)')
ValueError: Not naive datetime (tzinfo is already set)

According to this:

celery/celery#983

It seems to be a bug?

Not installing

Hi I get the message that it is not installed becouse there is no geopy, but I have installed geopy.
Any idea how to sove this?

C:\Users\armh>conda install -c iannesbitt readgssi
Channels:
 - iannesbitt
 - defaults
Platform: win-64
Collecting package metadata (repodata.json): done
Solving environment: failed

LibMambaUnsatisfiableError: Encountered problems while solving:
  - nothing provides geopy needed by readgssi-0.0.17-py37hd844fa7_0

Could not solve for environment specs
The following package could not be installed
└─ readgssi is not installable because it requires
   └─ geopy, which does not exist (perhaps a missing channel).

How to load gprMax simulation files?

I used a gprMax GSSI simulated output file and gprMax's MATLAB .out to .dzt converter to create a dzt file based on my simulation.

However, when I try to read this file with readgssi -i PATH_TO_FILE.dzt I run into this error:

I cannot find similar issues, so I'm posting this one. How can I resolve this?

rh_system and rh_version are not properly read

the versioning and system byte (byte offset 113 in header) is not read properly.

readgssi/readgssi/dzt.py

Lines 154 to 155 in 72b5e78

	header['rh_version'] = ord(vsbyte) >> 5 # whether or not the system is GPS-capable, 1=no 2=yes (does not mean GPS is in file)
	header['rh_system'] = ord(vsbyte) >> 3 # the system type (values in UNIT={...} dictionary above)

must be changed to:

    header['rh_version'] = int('{0:08b}'.format(ord(vsbyte))[::-1][:3], 2) # whether or not the system is GPS-capable, 1=no 2=yes (does not mean GPS is in file)
    header['rh_system'] = int('{0:08b}'.format(ord(vsbyte))[::-1][3:], 2) # the system type (values in UNIT={...} dictionary in constants.py)

Problem with date and time when reading .DZG file

Hello
I ran into this error for the first time when reading .DZG files:
ValueError: Not naive datetime (tzinfo is already set)

As soon as I move the .DZG files into another directory, the code runs and outputs the .csv files for the GPR data, I'm wondering how to fix this. I've emailed the files to [email protected].
Thank you!

GSSI 4105NR frequency 2GHz and others

Hello Ian,

I have tried your code to independently convert dzt to CSV files and I was very impressed. Thank you for putting this together and sharing it.

One issue: I am using a GSSI 4105NR air-coupled antenna with a central frequency of 2000MHz. This particular antenna is not included in your dictionary.

Second question: I was able to normalize the distance based on the GPX file. How would you that if you want to normalize based on the DMI file?

Third question: Is it possible to split files in smaller ones based on distance? (Not discussed in the manual)

Thank you in advance for your response.

Eyoab