openhd / qopenhd

QOpenHD App

QOpenHD

QOpenHD is the default OpenHD companion app that runs on the OHD Ground station or any other "external" devices connected to the ground station.

It is responsible for displaying the main video stream to the user, composed with the OSD, and changing OpenHD settings.

As the name suggests, it is based on QT (5.15.X) and will not run on older versions.

Developer Design Overview (incomplete)

1. QOpenHD is not OpenHD (main). It can communicate with a running OpenHD main instance (ground and/or air) via Mavlink and receives the (primary / secondary / ++) video streams.
2. While QOpenHD is the default companion app, OpenHD MUST NOT assume there is a QOpenHD instance somewhere that initiates voodoo settings / setup processes. This is in contrast to OpenHD/QOpenHD releases before the "evo" series. As a result of this limitation, both OpenHD and QOpenHD can be developed independently from each other, making debugging easier.

Installing

Like every OpenHD app or module, we publish packages into our Cloudsmith Repository. There are Packages for X86 (ubuntu 22.04,23.04), armhf (rpi, arm64 rockchip). Android releases are available on the Playstore and can be downloaded from there.

Compiling

We have a CI setup that checks compilation on Ubuntu. You can review the steps it performs to build and run QOpenHD. Other platforms than Linux are not supported at the moment.

Building QOpenHD Locally in QT (CURRENTLY UNDER DEVELOPMENT)

Mac

1. Install Xcode from the Mac App Store.

2. Install Qt using the Qt online installer.

3. Download Qt 5.15.0+ for Mac and Qt Creator using the Qt Installer.

4. Clone the source code:

   git clone --recurse-submodules https://github.com/OpenHD/QOpenHD.git
   

   If you use a GUI git client, ensure you update the submodules or QOpenHD won't build properly.

iOS

1. Install Xcode from the Mac App Store.
2. Install Qt using the Qt online installer.
3. Download Qt 5.15.0+ for iOS and Qt Creator using the Qt Installer.
4. You'll need an Apple developer membership to install directly on an iOS device.
5. Clone the source code as mentioned earlier.

Windows (CURRENTLY UNDER DEVELOPMENT)

1. Install Visual Studio 2019 (free).
2. Install Qt using the Qt online installer.
3. Download Qt 5.15.0+ and Qt Creator using the Qt Installer.
4. Download the GStreamer development kit for Windows, both the Runtime and Development packages (32-bit MinGW packages, version 1.14.4). QT will look for gstreamer in c:/gstreamer/1.0/x86.
5. Clone the source code as mentioned earlier.

Linux

QTCreator

1. Install Qt using the Qt online installer.
2. Have it download Qt 5.15.0+ for Linux.
3. Install GStreamer development packages from the package manager (e.g., on Ubuntu, run apt install gstreamer1.0-gl libgstreamer1.0-dev libgstreamer-plugins-good1.0-dev gstreamer1.0-plugins-good libgstreamer-plugins-base1.0-dev gstreamer1.0-plugins-base libgstreamer-plugins-bad1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-ugly gstreamer1.0-qt libxcb-cursor0).
4. Clone the source code as mentioned earlier.

Manual via qmake

1. Clone the source code.
2. Run 'bash install_build_dependencies.sh ARCHITECTURE' (ARCHITECTURE can be X86, rpi, rock5).
3. Run 'build_qmake.sh'.
4. Find the binary under build/releases/QOpenHD.

Android

1. Install Android Studio.
2. Install Android SDK level 28 and NDK r18b using Android Studio.
3. Install Qt using the Qt online installer.
4. Download Qt 5.15.0+ for Android (not for the OS you're building on).
5. Clone the source code as mentioned earlier.

Building QOpenHD

Step 1) clone this repository with --recurse-submodules Step 2) install all dependencies Step 3) recommended - open in QT creator. Otherwise, you can build QOpenHD via the command line:

mkdir build
cd build
qmake ..

Contributing

Thanks to all the people who have contributed !

qopenhd's Issues

Some settings don't save properly

Specifically those represented by text fields, and a few in the "other" tab of ground pi settings.

The frequency setting didn't seem to actually save when I tried it, even though other settings save just fine. That may be due to the way the text field settings are handled.

However some of the other, less common settings ended up being written incorrectly into the openhd-settings-1.txt file on the ground side, which prevented the ground pi from booting again. I'm not sure what the cause of that one is yet.

Telemetry protocols

Mavlink is the primary telemetry protocol, but there are others that could be supported easily.

Classes are added for a few, but none of the others are implemented yet.

---

Status

• Mavlink
• FrSky
• LTM
• SmartPort
• Vector Open Telemetry

Improved Recording features

The project could benefit from improved and mature recording features. It is openHD after all so should be all about image and image quality.

1. in qopenhd an actual "rec" button/control. This should be configurable between
   -screen recording of the current display
   -recording the recieved video stream only
   -recording the onboard (higher quality) video on the airpi
   -perhaps an option to do all of the above simultaneously
2. screenshot "stills"
3. record features should be possible to trigger via controller/transmitter
4. video storage space remaining
5. perhaps quality/resolution of the airpi recording
6. wishlist item (1) share to youtube. which will also promote project
7. wishlist item (2) live stream
8. wishlist item (3) airpi recording to external usb drive.. probably not used much as I think most people fly a pi zero or 3a+

Warn the user when vehicle is going out of range or losing RC control

The simplest UI would be to turn the upper bar yellow (it's currently red when the vehicle is armed), and announce it with the speech system.

As for the way to decide whether the vehicle is about to go out of range or losing RC control, that is a little more involved but could be a combination of RSSI, distance, and perhaps the rate of lost packets.

Ground Pi periodically does not retain the correct settings after smart sync or reboot.

Justin Healey: Stephen,

On Qopenhd, I changed the FEC and block size parameters a couple times and there was significant packet loss. I was a little concerned at first thinking something might have broke but, what seems to keep happening is:

1. Save Parameters on Ground.
2. Reboot w/ smartsync (Binding).
3. Params save to the Air Pi fine.
4. For some reason, the ground reverts to previous FEC/Block even after another reboot. Air seems to retain the correct settings just fine.

*also on one occasion the same thing happened when I tried to change the channel, it was transmitted and saved on the airside just fine but, the ground was stuck at 2484 I ran the same test several times with the old Openhd settings app which never had that problem and it still seems to work fine.

Obviously the concern is that FEC/Blocksize is a parameter that is modified quite regularly by users and because the system requires both air and ground to have the same settings it could result in poor performance without users realizing that they don't match.

I will try to gather more data but, the best way to test this out is to change some of the video or Radio Transmission params, run the synchronization and, another full reboot of each side just to be safe (Especially Atheros Cards).

Then, if you notice the channel you selected is not correct or, you see packet loss, individually remove and diff the air/ground settings.txt file to see where the mismatch is.

Vitaliy was originally extremely specific about the ReadOnly stuff because he encountered problems just like this and was unable to fully resolve them.

I'm not sure if we're dealing with the same problems or not.

Edit

I also encountered 3 situations where Kernel panic happened, & Settings were completely wiped twice...

Ath9k Settings

Hello,

Great work so far!

Can we move Atheros settings to "Other" tab or better-yet, an "Advanced tab" because it is something we don't want people who don't know what they are doing screwing around with.

Thoughts?

-Htcohio

Make OSD widgets drag-and-drop

The current widgets are in static locations, they can be enabled/disabled but not moved.

They need to be switched over to a grid covering the whole screen area, and then drag-and-drop needs to be enabled with the individual locations of each widget persisted with QML Settings.

Air/Ground ATH9k_htc carrier sense medium access on QOpenhd

The RC uplink that is primarily used with atheros cards could benefit from having easy access to all of the other parameters in the link (other than TX power because that's already included in the main settings page).

We would need fields within the QOpenhd app to define air and ground independently.
.. then, SSync uploads the new parameters to the airside after reboot.

https://github.com/HD-Fpv/Open.HD_Image_Builder/blob/master/stages/05-Wifibroadcast/FILES/overlay/etc/modprobe.d/ath9k_hw.conf

Battery gauge color warning/error levels are hardcoded

The battery gauge has a setting for the number of cells but the gauge color ignores it and assumes there are 3 cells.

Flight timer widget not enabled

Just needs to start counting when armed is set.

Move Enable Q OSD option to first page

Steve,

Right now, enable Q OSD is under "OTHER" settings.

Since Old or New OSD option with HDMI output is likely to be used a lot, what do you think about moving it to the top-left app tab?

False Arm Detected When QGC is also Connected

App interface flashes red indicating arm condition when qgc is also connected. Suspect some bleed over from qgc heartbeats.

Set boot partition read-only

The ground station partitions are currently read-write to allow QOpenHD to save and load its own settings when it needs to, and in general setting the partitions read-only can't prevent most SD card problems.

However, in the specific case of the openhd-settings-1.txt and osdconfig.txt files, which are on the /boot partition (fat32 filesystem), there may be cause to keep it read-only when it isn't being edited. This requires further testing and may or may not solve anything, but QOpenHD doesn't need it to be read-write so it can't hurt to try.

App can be started more than once on some platforms

Just requires using QSingleApplication or a simple shared memory check (3 lines of code) to prevent starting the app twice.

This can't happen on Android, Mac, or iOS, it can happen on Linux though.

Mouse pointer disabled on ground station

The mouse pointer is currently disabled on the ground station, making it difficult to use a mouse or other pointing device other than the touchscreen.

Disabling it is intentional but may not be necessary, and if it can't be re-enabled entirely we should detect that a mouse is connected and enable it dynamically.

Mavlink doesn't work on GroundPi at first boot

When QOpenHD is started at boot on the GroundPi itself, the mavlink fifo seems to be stuck. It works fine when is run manually via ssh.

Also seems to cause other mavlink listeners to get stuck until the app reads from the fifo, including remote apps connected via UDP.

Add sky view of GPS satellites

It would be cool to have a view of where the satellites are in the sky overhead, and might prove useful in some cases. You would be able to see if they're not spread out as much as they should be (affects horizontal accuracy, apparently), and you could see if something is blocking line of sight or not in a particular direction.

Settings take 10 seconds to load

This is caused by the GroundPi not having a delimiter to signal that all settings have been sent to the app.

The app checks for a ConfigEnd=ConfigEnd UDP packet, so that it knows it has received all the ground station settings. Otherwise it waits 10 seconds to be sure they weren't delayed as there is no other way to know all of them have been received. This makes it look like the settings panel is just disabled for 10 seconds, even though there is an activity spinner (not visible at the moment unless the GroundPi tab is selected).

This can be trivially fixed with two lines added to the beginning and end of this function in RemoteSettings.py on the GroundPi side:

def SendAllSettingToPhone():
    global IPAndroidClient
    for te in complete_response['settings']:
        
        SendBuff = bytearray()
            #Header
        SendBuff[0:0] = ConfigResp

    
            #Form Payload. Header from 0 to 19 byte, 20+ data
        ValuePayload = te.encode('utf-8')
        SendBuff.extend(te.encode('utf-8') )
        SendBuff.extend(str.encode("=",'utf-8') )
    
        data = complete_response['settings'][te]
        ValueDataPayload = data.encode('utf-8')
        SendBuff.extend(ValueDataPayload)

        SendData(IPAndroidClient,SendBuff)
        print("v :", SendBuff)
        SendBuff.clear()
        sleep(0.02)
    SendData(IPAndroidClient, bytearray(b'ConfigEnd=ConfigEnd'))

Possibly no video with different Block/FEC params

Hey Steve,

Today I had a short flight with the following parameters.
• Datarate= 3
• Blocksize= 1550
• Video Blocks= 7
• FEC Blocks= 2
• Resolution 720p 30FPS
• Keyframes = 10

I was not able to get any video up on QOpenHD,

I had telemetry but video wouldn't display, I power cycle twice and couldn't figure out what was going on.

Qgroundcontrol was working fine but even that was a little choppy more than I would expect.

It wasn't until after the fact that I realized the block size and FEC parameters I was using could have caused a problem in the app.

Everything looked fine on the native display.

I will try to double-check everything but, the next time you power the system up maybe you could confirm whether it was isolated on my side or if something was stopping the video with those settings.

Cheers

Enable RC

The RC system is currently disabled on some platforms by a compiler flag due to differences in how joysticks work on those platforms and for safety reasons (it must be tested on every platform).

Platforms where remote RC is currently enabled

• Mac
• Windows
• Android
• Linux
• iOS

Autohide for PiP may not be working correctly everywhere yet

Use WorkerScript for long running or looping QML code

There are parts of the UI layer, particularly the settings handling, that require long running functions or loops. These have the potential to interrupt the main thread and cause problems with animations or other rendering.

They should be moved to a QML WorkerScript in the short term until/unless the settings code is moved to C++.

Local widget settings default to disabled

Commit e1d1b07 changed the local settings to use bindings instead of explicit calls, which seems to be causing the settings values to be saved to the default value (false/off) of the switches at startup.

Reverting that commit should fix it.

Add separate overlay panels for additional statistics

Radio FEC error count, dropped packets, battery details, ground/air CPU/temperature graph, etc

Add widget enable/disable switch to the popover for each one

The setting to enable/disable widgets is "buried" in the settings panel at the moment. It's not hard to find and they're all in a nice list, but it would be nice to just tap them and be able to turn them off too.

Enable hardware accelerated video decoding/rendering

The QtGStreamer video code is going to be replaced with qmlglsink, which is a modern and officially supported part of GStreamer.

This dramatically simplifies the code and puts responsibility for constructing the GStreamer pipeline inside the app itself. This allows us to do things like easily swap in other video sources, for example the videotestsrc component of GStreamer, which we're going to use to show a "video offline" screen whenever the real camera stream stops for whatever reason.

The GStreamer androidmedia plugin is not currently loading due to some additional build requirements but that will be resolved soon.

Display home distance

The home distance widget isn't enabled at the moment.

Translate UI text, localize numbers

Some spots in the code are already using the Qt translation system even though there are no translations added yet, and some are currently localizing number formatting (lost/damaged packet counters).

This issue tracks which parts of the UI have translations enabled.

• Settings sidebar
• Settings buttons
• App settings tab names
• App settings row titles
• GroundPi settings tab names
• GroundPi settings row titles
• Status panel text
• GPS widget
• Downlink widget
• Uplink widget
• Air status widget
• Ground status widget
• Bitrate widget
• Home distance widget
• Flight timer widget
• Flight mode widget
• Battery widget
• About panel (license mostly)

Add system for saving/loading settings

I propose having a way of saving or loading presets in a sharable format.

This will make it easy for users to easily share configurations considering the vast amount of hardware options that have not been fully tested.

Last Mavlink heartbeat value is wrong if drone has never connected

Custom Video Framerates

The Raspivid Framerate Options 30, 48 and 59.9 are the standard options however, some situations require custom framerates.

Examples:

CSI-HDMI adapter.... Some HDMI cameras only have PAL framerates of 24/25FPS.

IP Cameras at 30fps require 240FPS HelloVideo.bin player for the best playback

Settings are being saved even when they haven't been modified

For some reason, ground pi settings are being marked as though they've been modified even when they haven't causing the settings system in the app to save them.

This isn't inherently a problem as long as the settings system is working properly, but it's not necessary either.

Change frequency setting UI to a dropdown list

Currently a text field, needs to be a dropdown so typing isn't required and to prevent incorrect values from being entered.

Remote drag-and-drop does not work when using USB tethering

Remote drag-and-drop currently hardcodes the ground station address to guarantee that it works in the most common case with Android connected to the WiFi hotspot, however this inherently does not work when USB tethering is in use.

The solution is to use the ip address sent by the ground station to the settings port, which is already being used to make settings work with USB tethering.

Add ground station settings detail info for tooltips

v0.3.0 added tooltips for each ground station setting, the information for them comes from the settings model/mapping. Each setting in the mapping can include an optional field named "info". The text in that field will automatically show up in the tooltip, and if a setting doesn't have one then "N/A" is currently shown.

The only settings that have an info field at the moment are DISPLAY_OSD and ENABLE_QOPENHD.

The tooltip area can be increased if it proves to be too small for the information we need to display there.

Air/Ground ATH9k_htc carrier sense medium access on QOpenhd

The RC uplink that is primarily used with atheros cards could benefit from having easy access to all of the other parameters in the link (other than TX power because that's already included in the main settings page).

We would need them for air and ground independently.
*So, save modprobe/ath9k for both air and ground on the ground side.. then, SSync uploads the new parameters to the airside.

https://github.com/HD-Fpv/Open.HD_Image_Builder/blob/master/stages/05-Wifibroadcast/FILES/overlay/etc/modprobe.d/ath9k_hw.conf

Use column QML type for app settings

The current QML code for the app settings rows has the total height and the vertical offsets for each row hardcoded. It's quite fragile.

Those row layouts should use the column type and remove all of the manual layout.

Move each widget to a separate QML file

Will make it much easier to deal with them, and it'll be a good first step toward moving them all to a drag-and-drop grid.

Move settings activity indicator to the list view

When settings are being loaded or refreshed, the activity indicator in the lower right corner of the settings panel starts animating.

With pull-to-refresh added, this is unintuitive.

The activity indicator should be moved to the top of the list view in the header area (which should be hidden unless refreshing) and the settings rows should be cleared before a refresh.

Move settings processing down to C++ layer

The code that handles mapping the settings values and organizing them into models is currently in the QML layer. The code is pretty fast but by definition it's a loop of almost 100 items and it happens in the main thread, which on occasion causes hiccups in the UI. It's most noticeable when the settings window first opens and closes.

The settings maps and the associated processing code should be moved down to the C++ layer and run in a separate thread.

Settings panel does not show an indicator of success when saving settings

The settings panel currently saves silently, there is a separate branch with changes to show a confirmation but it is not finished and merged yet.

Replace gstreamer with tuned decode/render pipelines

GStreamer has a number of quirks and bugs, particularly in specific versions. In particular, the version we use in QOpenHD has a known issue with rendering in QML that causes frame drops and jitter to appear, despite video being received, buffered, decoded and delivered to qmlglsink on time and consistently.

GStreamer also greatly complicates the build process and has a huge number of dependencies it ships with that have to be linked to in the correct order and/or shipped with the app, in part to ensure that most of the elements that decodebin might automatically decide to use, will actually work.

I've written a replacement decoder/renderer already, which has been tested by Justin and Thomas and seems to work great. The new video renderer closely mirrors what hello_video does on the ground station but on other platforms, feeding h264 data directly to the available hardware decoder and letting it render directly to the screen (aka zero copy, no decoded frames handled in the app itself).

Current status on each platform:

• Android
• Windows
• iOS
• Mac
• Linux
• Raspberry Pi

Stream latency on the Pi

When running on the ground station itself, the video stream goes back through another gst-launch process and is forwarded to a UDP port due to an odd bug in filesrc. The video is also rendered using QtGstreamer which needs to be replaced. Both of those probably add latency.

Pi image - interface does not respond to touchscreen, mouse is working but pointer is invisible

Hi,

First of all thank you for all your work. I installed the Pi 0.3.1 image and it is working great except 3 minor issues, one being the flight mode not working on the QOHD Osd and the other 2, maybe related, the touch events have no effect on the OSD and mouse works but pointer is invisible, so it is a hard guess of actual screen position to enter the menu and operate changes. My touchscreen is working otherwise (other SD with graphical OS) and also using the QOHD image, through SSH, I can see touch events with evtest, so touch drivers are installed and working. I did not change anything else from the default installation but if needed I can attach logs.

Move telemetry string formatting to UI layer

The current telemetry code in c++ constructs strings with localized formatting for a number of properties, while the actual values are not all exposed as properties.

This was simple in the early stages of development, but has required duplicating some of them if the UI layer needed the value itself instead of a string for display, such as the heading or the battery voltage.

Those properties in the OpenHD class should be replaced with value properties and the localization and string formatting should be moved to the UI layer.

Remove redundant settings

Some settings are relevant both on the ground station side and inside the app itself, like the video port(s).

Currently the app itself does not use the ground station settings to configure its own behavior, but it should.

For example the rtp/raw setting and the video ports don't need a local setting, the app can just as easily read the ground station settings and configure itself based on those rather than duplicating them.

Pull-down "Refresh Params"

It seems as if we need a way to refresh parameters in the settings on the app.

In my opinion the customary way this is done on other apps or web pages is to simply pull down while in icon spins importing params.

Move map settings to a panel inside the larger map widget

The little popover isn't going to be enough for the map settings, and it's already a little cramped (the default map type dropdown styling is cutting off the text already).

Once the settings are moved then a single tap on the small map widget can be used to open the larger map widget instead of the popover.

Video latency/jitter on some platforms may require platform-specific overrides

In v0.3.0, the GStreamer pipeline is the same on every platform:

udpsrc port=5600 caps="application/x-rtp, media=(string)video, clock-rate=(int)90000, encoding-name=(string)H264" ! rtpjitterbuffer latency=25 mode=0 ! rtph264depay ! h264parse ! decodebin3 ! glupload ! glcolorconvert ! qmlglsink name=qmlglsink sync=false

When hardware decode is turned off, decodebin3 is replaced with avdec_h264, which is always a software decoder.

Because decodebin3 uses autoplugging to find available codecs and include any necessary elements to use them together, it may not always configure the pipeline the way we want it to.

Additionally, the vtdec_hw (Mac/iOS) hardware decoder seems to introduce about 500-1000ms of extra latency simply from including it in the pipline, regardless of how rtpjitterbuffer is set. This is likely due to an extra buffer somewhere but I haven't found it yet.

Add tooltip buttons to setting rows

It would be helpful to be able to tap a button on the settings rows and see more information about individual settings.

The easy way to do that is to add a Button instance to the delegates with a Tooltip instance inside it