sco1 / ppg-log

Helper Tools for PPG FlySight Flight Log Management

Hi there, I'm sco1! 👋

I'm a Boston-based Aerospace Engineer focused on test & evaluation.

Programming Interests

• Data collection, postprocessing, and visualization
• CLI and GUI tooling
• Code quality
• Software testing

I also find joy in making hilarious things like:

• randomly-change-self - Change all instances of "self" to "this" in one randomly chosen scope
• pytest-pahrametahrize - Parametrize your tests with a Boston accent
• brie-commit - A collection of cheesy pre-commit hooks
• pre-commit-fail-randomly - Randomly fail a pre-commit run

ppg-log's Issues

Add CLI

Deduplication failure for batch insert to empty database

The current deduplication logic in db.bulk_insert (see: 6191402) only checks for existing entries in the database itself, so we'll still fail the unique constraint if there are duplicates within the batch to be inserted.

Add flight summary database

Start simple with something local (sqlite?)

Desired per-log metrics:

• Flight date/time
• Total takeoffs
• Total landings
• Flight segment length(s)
• Total flight time

DB Insertion fails for flight logs without processed flights

Fix in FlightLogEntry.from_flight_log

Add capability to process in-fight logging startup

For cases where we forget to turn the logger on before taking off, there should be an ability to begin a flight segment at the start of the file. Since there was an intentional decision to not consider segments at the start of the file (see: #5), this should be an opt-in switch.

While we're at it, the default behavior should fail more gracefully rather than conking out in the segment classifier when attempting to reshape the segment start/end indices.

Add Support For Flysight V2 Tracks

The new Flysight V2 hardware revision has a different log storage story & slightly different track file formatting. See: sco1/pyflysight#8 for a description of the changes.

It might be better to just lean on pyflysight once the parsing is finished rather than duplicating the parsing effort.

Flight segments not detected if intermediate spikes are present

A side effect of the fixes made by #4 introduced a different bug I didn't notice while testing the fix. As shown in the below plot, if intermediate spikes trigger the merging logic but aren't close enough to a valid flight segment to actually merge, it prevents identifying any valid flight segments later in the flight:

Summary query fails for empty DB

Incorrect flight segmentation with landing instability

While the flight segment detection logic from b20f785 seems to do a good job mitigating noise on takeoff, it does not correctly account for noise on landing.

In this example, the start of flight segment 2 is incorrectly defined in the noisy region of flight segment 1's landing:

While the ultimate fix may be a side effect of the logic for #1, as a starting point we probably need to use a time threshold between segments to determine when to turn off the segment merging flag rather than just assuming that the landing data is clean.

Add GPX and/or KML Parsing

For those times when we forget to bring our Flysight or we do bring one and the battery is dead, we might have a backup that needs parsing!

Seemingly valid flight segments not detected

Not entirely sure why flight segments are being missed for these logs:

Improve flight segment detection logic for touch & gos

The time-based logic for identifying flight segments added by b20f785 does decently well for the general flight case but is hit or miss on detecting touch & gos since it's only relying on total velocity. For example, the final segment of this flight log has two touch & gos that are not identified because the total velocity never drops below the "on ground" threshold (they're also very short ground segments):

This might be improved by adding an altitude consideration into the determination of the inflight vs. ground classification and/or the determination of valid flight segments? It would still be difficult to differentiate a touch & go from a low pass, where we're only a few feet off of the ground but not in a landing phase, especially if there is a decent amount of ground wind. Perhaps a time component could also be used.

On the other hand, perhaps it's not worth the added (possibly fragile) complexity and we instead just call the existing logic "good enough" as a tool for determining total flight time & maybe keep touch & go classification as an experimental feature.

Incorrect handling of start-of-file noise

An attempt was made in b20f785 to mitigate noise at the beginning of the file causing an incorrect flight start by just trimming some data from the beginning of the file. Unfortunately, this didn't quite fix the problem, at least with the current default:

While the assumption that the FlySight will be on the ground when it's turned on is correct, we need a better way to discard segments that start very near to the beginning of the file. This may be better served with a longer trim default as a quick fix? It's also worth looking at the satellites in view to see if this is just from a lack of accuracy on startup.

I also think this threshold would be better in the flight segment classifier rather than just trimming off the beginning of the data file.