elvinc / gyroflow Goto Github PK

Hi, @ElvinC
Very amazing work. I'm interested in the stabilization method used in your work, it seems that you just apply a low pass filter to the raw gyro data, and do slerp on the integrated orientation, I just wanna what's the principle of this method, especially the slerp operation, as far as I understand, slerp is just a method of interpolation applied to two quaternions, how could it stabilizes the jitter of videos? What's more, how well does this stab method work? Or maybe the good results mainly benefits from the strict camera calibration part?

Hi! I found tool to convert ardupilot logs to CSV format. In mission planner, CTRL+F -> Dashware -> Select .bin log file -> fill only IMU in text field.

Here example converted log file:
https://1drv.ms/u/s!AuFPVI_eDaZbhY4X3UuhfYfvd-SgFg?e=ox3Lfe

i think IMU_GyrX,IMU_GyrY,IMU_GyrZ already in radians
-0.0003537596 | -0.0006091184 | 0.0002790984
It would be great to add ardupilot support to the program!

At first I thought it was was older non supported cameras. Quickly realised that it is not.
So I presume it is for older versions of the software?
Is it no longer supported by newer builds?

Maybe change the folder name?

Hello. I am wondering that, does gopro provide real time sensor and video data?
It looks like people use the log and already dowloaded data.
I want to use Gopro for real time visual odometry purpose for mobile SLAM project.
If we can get the sensor data it will be really good. Thank you for reading it.

Not sure if this is a bug, or that INAV blackbox logs are just not supported. If the latter please considere this a feature request ;).

With that out of the way, I am trying to use an INAV blackbox log to stabilize some video. INAV blackbox logs are .txt files, so I converted it to .csv using cleanflight/blackbox-tools. When trying to run the synchronization Gyroflow throws the error ValueError: 'gyroADC[0]' is not in list. gyroADC[0] is in the csv file though. Any help would be much appreciated. Thanks!

Have you used insta360 studio? It basically applies gyro correction realtime and allow you to tweak some stuff. Maybe we can be able to embed the gyro data into the video in the format insta360 studio expects and let it do the magic?

Right now on sharp moves video may be moved too much, so black borders will be seen at any crop. Please add configurable limit for video movement

fixed issue.

i was right clicking and "saving link as"

Hi, @ElvinC
I am trying to apply your program to mobile phones, now I have recorded the video and the corresponding gyro data, but I'm not sure if the format of the gyro data is correct. So could you please offer an example of gyro data and the corresponding video as one demo for checking what the problem is. If so, I would appreciate it a lot. Thank you very much.

INav blackbox converter adds whitespace to format CSV. I have to manually remove whitespace from headers in order for gyroflow to accept logs

Any suggestions how to get it working? I am on ubnutu, 20.04.3 LTS and get this one:

mindaugas@tux:~/Downloads/gyroflow-master$ poetry run python3 gyroflow.py
Traceback (most recent call last):
File "gyroflow.py", line 11, in
import calibrate_video
File "/home/mindaugas/.cache/pypoetry/virtualenvs/gyroflow-iray_dlu-py3.8/lib/python3.8/site-packages/shiboken2/files.dir/shibokensupport/feature.py", line 142, in _import
return original_import(name, *args, **kwargs)
File "/home/mindaugas/Downloads/gyroflow-master/calibrate_video.py", line 6, in
from tkinter import Tk
File "/home/mindaugas/.cache/pypoetry/virtualenvs/gyroflow-iray_dlu-py3.8/lib/python3.8/site-packages/shiboken2/files.dir/shibokensupport/feature.py", line 142, in _import
return original_import(name, *args, **kwargs)
ModuleNotFoundError: No module named 'tkinter'

I have created a lens preset file for the Gopro Hero 8 Black and had submitted it via the Gyroflow lens preset form ( https://forms.gle/6iKrUQWqCFxVghLd6 )about a week ago but it has not been added to the presets list yet.

My Sony_Action_Cam_HDR-AS30V preset was uploaded much faster and so I assume the dev's are busy.

Since the Gopro 8 is a popular camera and there is no official preset available, I am attaching the preset I created here in case anyone needs it before we have an official one added to the repository.

Please remove the .txt extension as I am unable to upload .json files to this message.

GoPro_Hero_8_Black_2_7K_4by3_60fps_wide.json.txt

Hi,
Thank you for your awesome work !!!
I'm going to play with it, but some package is missing i think.
Which one

(cv_cuda) PS C:\Temp\gyroflow-master\gyroflow-master> python.exe .\gyroflow.py
Traceback (most recent call last):
File ".\gyroflow.py", line 13, in
from stabilizer import GPMFStabilizer
File "C:\Temp\gyroflow-master\gyroflow-master\stabilizer.py", line 8, in
from blackbox_extract import BlackboxExtractor
File "C:\Temp\gyroflow-master\gyroflow-master\blackbox_extract.py", line 1, in
from orangebox import Parser
ModuleNotFoundError: No module named 'orangebox'

Thank you in advance,
Kris

Basically the title. I think the lens correction works great, and I would like to render a video that corrects the distortion without adding any stabilization. Right now Gyroflow assumes not providing a bblog means the gyro data is embedded in the video.

Thanks!

AVG is reporting this contains malware. It's probably a false positive.

Hello,
I've just cloned the repo and installed a few python packages with pip.
I wanted to try your code with a GoPro session 5 clip (I had to change the file extension from "MP4" to "mp4").

I set up the UI like this:

All the processing seems to complete successfully, but when I hit export I have this error in the terminal (I also tried various combinations in the nearby checks and bitrate):

17:33:56 :: WriteGear :: ERROR :: BrokenPipeError caught, Wrong values passed to FFmpeg Pipe, Kindly Refer Docs!
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/vidgear/gears/writegear.py", line 315, in write
    self.__process.stdin.write(frame.tostring())
BrokenPipeError: [Errno 32] Pipe interrotta

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "gyroflow.py", line 1519, in export_video
    self.stab.renderfile(start_time, stop_time, filename[0], out_size = out_size,
  File "/home/samuele/GIT/gyroflow/stabilizer.py", line 571, in renderfile
    out.write(concatted)
  File "/usr/local/lib/python3.8/dist-packages/vidgear/gears/writegear.py", line 321, in write
    raise ValueError  # for testing purpose only
ValueError

I'm on a Linux machine running Linux Mint 20.1.

~ $ ffmpeg 
ffmpeg version 4.3.git Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04)
  configuration: 
  libavutil      56. 59.100 / 56. 59.100
  libavcodec     58.106.100 / 58.106.100
  libavformat    58. 56.100 / 58. 56.100
  libavdevice    58. 11.102 / 58. 11.102
  libavfilter     7. 87.100 /  7. 87.100
  libswscale      5.  8.100 /  5.  8.100
  libswresample   3.  8.100 /  3.  8.100

I hope this can be useful.
If any other info is needed, please let me know.

Thanks, bye.

Hi, can't run stabilization
OS linux ubuntu
Record test video on Runcam Hybrid 4k
Compute offset and make GPMF Emulated video file

after this open exported video file in latest gyroflow with this settings. Not calibrate camera yet, but set similar camera settings

at 30 seconds video shaking started
and this error i receive in console

Gyro rate: 396.2234948150441 Hz, should be close to 200 or 400 Hz
Starting sync. Smoothness: 0.85, sync1: 5.0 (frame 299), sync2: 30.0 (frame 1798), OF slices of 60 frames
Analyzing frame: 0/60
Traceback (most recent call last):
File "gyroflow.py", line 1452, in recompute_stab
self.stab.auto_sync_stab(smoothness,sync1_frame, sync2_frame, OF_slice_length)
File "/home/hyperion/gyroflow-master/stabilizer.py", line 24, in auto_sync_stab
d1, times1, transforms1 = self.optical_flow_comparison(sliceframe1, slicelength)
File "/home/hyperion/gyroflow-master/stabilizer.py", line 222, in optical_flow_comparison
thisrot = Rotation.from_matrix(rots[0]) # First one?
AttributeError: type object 'Rotation' has no attribute 'from_matrix'

I see there is a placeholder for Ubuntu packaging on the website. I tried in vain to follow the install, and after working around a few dependency errors, I finally gave up.

If you could provide either a deb or snap, it would be awesome.

Thanks for the (free) software, from what I see on the video and website, it looks great.

I would offer to help with the packaging, but I was never able to get my own copy working, and I'm new to Ubuntu, after decades on some form of Redhat/Fedora.

The GoPro Hero 8 and Hero 9 use H265 hevc codec when using "High" bitrate setting, and high resolution (2.7K 60, or 4K, etc).

Gyroflow doesn't seem to support it. When trying to use H265 footage from Hero8 or Hero9, I get the following error in the terminal when I press "Apply settings and compute sync":

Edit: h265 works. This error comes when using 12GB chapters enabled with Labs firmware.

Preset name: GoPro_Hero7__2.7K 4by3 wide
Note: 
Made with 13 frames using calibrator version 0.2.0-alpha on date 2021-02-06
Warning: Versions don't match. Calibrator: 0.3.0-beta/dev, Preset: 0.2.0-alpha
[warn] [/atom[2]/movie/atom[1]/udta/atom[0]/tag] Unable to convert string to Unicode: 'ascii' codec can't decode byte 0xa9 in position 0: ordinal not in range(128)
[warn] [/atom[2]/movie/atom[3]/track/atom[3]/media/atom[1]/hdlr] [Autofix] Delete '/atom[2]/movie/atom[3]/track/atom[3]/media/atom[1]/hdlr/name' (too large)
[warn] [/atom[2]/movie/atom[3]/track/atom[3]/media/atom[1]/hdlr] [Autofix] Fix parser error: stop parser, add padding
[warn] [/atom[2]/movie/atom[4]/track/atom[2]/media/atom[1]/hdlr] [Autofix] Delete '/atom[2]/movie/atom[4]/track/atom[2]/media/atom[1]/hdlr/name' (too large)
[warn] [/atom[2]/movie/atom[4]/track/atom[2]/media/atom[1]/hdlr] [Autofix] Fix parser error: stop parser, add padding
[warn] [/atom[2]/movie/atom[4]/track/atom[2]/media/atom[2]/minf/atom[2]/stbl/atom[0]/stsd/sample_entry[0]/atom[0]/esds/ES/decConfigDescr/reserved] padding contents doesn't look normal (invalid pattern)
[warn] [/atom[2]/movie/atom[5]/track/atom[1]/media/atom[1]/hdlr] [Autofix] Delete '/atom[2]/movie/atom[5]/track/atom[1]/media/atom[1]/hdlr/name' (too large)
[warn] [/atom[2]/movie/atom[5]/track/atom[1]/media/atom[1]/hdlr] [Autofix] Fix parser error: stop parser, add padding
<generator object get_payloads at 0x7f45a6719270>
Traceback (most recent call last):
  File "gyroflow.py", line 1966, in recompute_stab
    self.stab = stabilizer.GPMFStabilizer(self.infile_path, self.preset_path, gyro_path=self.gyro_log_path, hero=heronum, fov_scale=fov_val, gyro_lpf_cutoff = gyro_lpf)
  File "/home/kristjan/gyroflow/stabilizer.py", line 979, in __init__
    self.gpmf = Extractor(gyro_path)
  File "/home/kristjan/gyroflow/GPMF_gyro.py", line 20, in __init__
    for gpmf_data, timestamps in payloads:
  File "/home/kristjan/gyroflow/gpmf/extract.py", line 60, in get_payloads
    offset = stco["chunk_offset[{}]".format(idx)].value
UnboundLocalError: local variable 'stco' referenced before assignment

For stabilization you could try to use the VidStab library. The VidStab library is basically a two phase process for digital image stabilization, stage one is to estimate motion and step two is stabilizing. However since you already have the movement data (blackbox or GPMF) you can skip step one and parse them in stage two, as shown in their documentation. You would still need to integrate/convert the gyro values into relative motion in x, y and rotation and put them in a comma-separated file.

import numpy as np
from vidstab import VidStab

# Read in csv transform data, of form (delta x, delta y, delta angle):
transforms = np.loadtxt(TRANSFORMATIONS_PATH, delimiter=',')

# Create stabilizer and supply numpy array of transforms
stabilizer = VidStab()
stabilizer.transforms = transforms

# Apply stabilizing transforms to INPUT_VIDEO_PATH and save to OUTPUT_VIDEO_PATH
stabilizer.apply_transforms(INPUT_VIDEO_PATH, OUTPUT_VIDEO_PATH)

Good luck! Love the project idea, I have been thinking about the same for quite some time and might contribute if I get some time during the weekends 👍

EDIT:
As mentioned, I've been looking for this for some time, I might as well share some useful resources when it comes to other relevant implementations:

• VirtualGimbal (very promising): https://github.com/user1321/virtualGimbal/tree/master/post_processing_software
• Hybrid solution: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6111248/#!po=6.25000
• Nvidia research: https://research.nvidia.com/sites/default/files/pubs/2014-09_A-Non-Linear-Filter/bell_troccoli_pulli_eccv2014.pdf
• Stanford older research: https://graphics.stanford.edu/papers/stabilization/karpenko_gyro.pdf
• Matlab implementation: https://github.com/alexgo1/Video-Stabilization
• Rolling shutter removal https://www.cc.gatech.edu/cpl/projects/rollingshutter/

Hi guys, im wondering if you could optimize some functions with nvidia cuda?
It looks pretty easy to use but unfortunally i'm not a python developer.

https://developer.nvidia.com/how-to-cuda-python

Hey mate, awesome work on this project!

I'm trying to to use stabilizer.py on a clip made with a 3d printed FPV drone frame (lot of oscillations due to resonance) and a flipped naked Hero 8. Here is the video

Like with RSGO, i've some weird oscillations in the frame_out but in RSGO the option Flip gyro data fix it.

Here's an easy way to implement the option? How can I do it ?

Cheers

I know the black parts aren't really a background color but just empty space but it would be cool if we could fill it with a custom color.

This would make it nicer to post on places where you control the background color like on your own website.

Up until we get automatic zooming this will make it a little bit nicer to watch.

Hi,
I've just stumbled upon this project.
I Think it could be useful to extract gyro data from black-box logs.

what is your blackbox_extract.py expected to output exactly? might give it a look in the next few days.

I really like gyroflow, as I can use it on any camera but,
right now there's only one point which is better on Reel steady - automatic zooming/un-zooming and rotation of video if there's some peaks in movement (where's now we can see some black areas in video).
Do you think this can be implemented? ideally with some kind of slider with 0-100%
Also what about horizon lock (as flowstate stabilization on insta360?
thank you,

WizzX

Hey I stumbled upon your repo searching for interesting GoPro footage stabilization.
I have not tested your code yet, however I saw that you are also experimenting with the calibration of IMU and camera.
I released my personal exploration repository for calibrating GoPros: https://github.com/urbste/OpenImuCameraCalibrator
Maybe you'll find something useful in there ;)
Cheers
Steffen

On the Hero 8 and Hero 9, the gyro is in perfect sync with the video.
Video+Gyro sync should not be needed.

This applies to the Labs firmware, but maybe also the latest normal firmware?
Old firmware for Hero8 is not in sync (the firmware that came with the camera back in the day).

I encountered an issue with the GUI version of Gyroflow.

After proper sinchronization was achieved I was not able to render my video, despite no error given.

I'm wondering if there's a way to use algorithms used in this project for stabilizing videos in real-time using gyro data.

Hello, i was testing gyroflow and all went well but when the stabilization was at 98.5% i got No more stabilization data and I was unable to play the outputted generated file. Then I tried again and setting the video stop to 10 seconds less and the stabilisation process froze on 99.9 percent and the output file was unplayable as well. For some reason I cannot get playable video file. I am running it on mac.

Edit: so I tried to run it on windows as well and same problem so I am thinking it must be something with the video or the preset. I used the preset bundled in the release for hero 7 4:3 4k and i tried both with gyro from video and fc gyro data. I also tried random videos from hero 6 that are just me walking and it always gets stuck on the end. I also tried yuv420p but no difference

I managed to make it work adding this two parameters.

Hello,
I would like to export the video with a bigger resolution than the source.
The reason I want this is so I can redistort the video afterwards and preserve as much reslution as possible.

ref Raphael on discord, libx264 fails when trying to encode a video with odd sizes.

We can "fix" it by adding a filter to the pipeline adding a black line or cropping 1px in to make resolution "right" or warn user to use a better colorspace as its related to 4:2:0 as described here: https://stackoverflow.com/a/23614652

I tested on videotoolbox (h264 HW encoder on Mac) and it worked with as well as it worked with prores 444 (also for Raphael)

For more info on potential fixes here:
https://stackoverflow.com/questions/20847674/ffmpeg-libx264-height-not-divisible-by-2

Hi guys amazing job 👍

I was wondering if we feed gyro data and video feed Jetson Nano can handle realtime manipulation of the video so that we can get a clean video. I am considering it to use as fpv stabilizer to use in OpenHD https://github.com/OpenHD/Open.HD

Thanks

I have a Runcam 5 orange and downloaded your latest release. Any way to get a hand on the 0.3.0-dev version? I would love to test it.

Hi

In the pyproject.toml file the required opencv-python version is set to 4.1.2 . However apparently there's no wheels of that version for python 3.9 see here ... As poetry uses the latest possible python version 3.9 (python = ">=3.7,<3.10" in pyproject.toml) I had to change it to python = ">=3.7,<3.9" in order for poetry install to work.

So I guess either the python version or opencv version should be changed ? I didn't make a pr as I'm not sure what to do...

Environment:
Mac OS X 11.4
gyroflow 0.3.0-dev

STR:

1. Launch the app
2. Open fancy stabilizer
3. Select input file, camera preset, gyro data
4. Open gyro log type selector (this window will later appear) and choose whatever you need (RC5 csv in my case)
5. Go to sync/stab tab and press "Apply settings and compute sync"

AR:
No unneeded windows appear on screen

ER:
Gyro log type selector appears on screen

Workaround here is to go back to "Input" tab and open/close that selector again. But this is quite annoying.
The same happens with any windows you've opened before, including "Save" dialog (with selecting where to save exported video

Screenshots of the problem:

I'm running Debian 10 with KDE: installing gyroflow with poetry results in:

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx.

Aborted

If I install every dependency manually and try to open a video file:

nik@nik-deb-desktop:~/repos/gyroflow$ python3 gyroflow.py 
ffmpeg version 4.1.6-1~deb10u1 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 8 (Debian 8.3.0-6)
  configuration: --prefix=/usr --extra-version='1~deb10u1' --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 22.100 / 56. 22.100
  libavcodec     58. 35.100 / 58. 35.100
  libavformat    58. 20.100 / 58. 20.100
  libavdevice    58.  5.100 / 58.  5.100
  libavfilter     7. 40.101 /  7. 40.101
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  3.100 /  5.  3.100
  libswresample   3.  3.100 /  3.  3.100
  libpostproc    55.  3.100 / 55.  3.100
Traceback (most recent call last):
  File "gyroflow.py", line 1834, in open_video_func
    self.video_info_dict["time"] = int(cap.get(cv2.CAP_PROP_FRAME_COUNT) / self.video_info_dict["fps"]) 
ZeroDivisionError: float division by zero

It seems like opencv is reporting 0 fps. I'm using opencv version opencv-python-4.5.1.48

Hi!
very interesting project :)
I'd like to test it in Linux, running Manjaro (arch based) at the moment, so if you could port it would be great :)
or instructions to build it locally?

Hi, i keep getting a runtime error while trying to "apply settings and compute sync".

does anyone know what i'm doing wrong ?

When you choose your input filename as output filename you get warned about overwriting the file. If you accept nonetheless the file gets overwritten with the new partial file and stops exporting.

Console output

Output file: D:/git/FPV/videos/GH011159.MP4
Starting to compute optimal Fov
QCoreApplication::exec: The event loop is already running
Done computing optimal Fov
frame: 579, 0/1540 (0.0%)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000026ecce18080] stream 1, offset 0x9b3c22d: partial file
frame: 584, 5/1540 (0.3%)
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000026ecce18080] stream 1, offset 0x9b3c439: partial file
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000026ecce18080] stream 0, offset 0x9b3c637: partial file
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000026ecce18080] stream 0, offset 0x9b75de0: partial file
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000026ecce18080] stream 1, offset 0x9bac8b7: partial file
[mov,mp4,m4a,3gp,3g2,mj2 @ 0000026ecce18080] stream 0, offset 0x9bacac1: partial file
...

I get this error when I try to "Apply Settings and compute sync".

Log info:

10.0 degree uptilt
using raw blackbox file
Preset name: Caddx_Vista__16by9 DVR
Note:
Made with 10 frames using calibrator version 0.2.0-alpha on date 2021-02-06
Warning: Versions don't match. Calibrator: 0.2.1-alpha, Preset: 0.2.0-alpha
F:/OneDrive/Hobbies/Drones/Videos/2021-06-17 Aggressive with Flywoo/btfl_all.bbl
Opening F:/OneDrive/Hobbies/Drones/Videos/2021-06-17 Aggressive with Flywoo/btfl_all.bbl
Traceback (most recent call last):
  File "gyroflow.py", line 2026, in recompute_stab
  File "stabilizer.py", line 1035, in __init__
  File "blackbox_extract.py", line 9, in __init__
  File "orangebox\parser.py", line 91, in load
  File "orangebox\reader.py", line 75, in __init__
  File "orangebox\reader.py", line 142, in set_log_index
  File "orangebox\reader.py", line 213, in _build_field_defs
RuntimeError: No predictor found for 10

On the dev branch, I had to downgrade opencv-python to run gyroflow.

This version doesn't work: 4.5.1.48 (from lock file)
This version works: 4.1.2.30 (changed in 4a5237f)

I'm using Ubuntu 20.04.
The error with [email protected]:

$ poetry run python gyroflow.py                                                                            
QObject::moveToThread: Current thread (0x2b88af0) is not the object's thread (0x2c38780).                                                                                                                            
Cannot move to target thread (0x2b88af0)                                                                  
                                                     
qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/home/kristjan/.cache/pypoetry/virtualenvs/gyroflow-oJ6C59RA-py3.8/lib/python3.8/site-packages/cv2/qt/plugins" even though it was found.
This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.
                                                     
Available platform plugins are: xcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx, webgl.                                               
                                                                                                                                                                                                                     
Aborted (core dumped)

Is there a way to create lens presets for smartphones? Like iPhones, Android phones, etc. Do they record gyroscope data with video? How would that kind of setup work?

It would be great to be able to automate the process of stabilizing by running gyroflow without a GUI, and passing all options in the cli. Supporting all options via cli could be hard, so using config files will probably be needed for more advanced options.

I almost always use the same settings (on first run), so it would speed up the process a lot.

Then one could also stabilize multiple files at once (in series), so I could stabilize 20 videos over night for instance.

How I imagine it'd work from the cli:

# cli options
gyroflow --smoothness 20 --zoom 40 --camera hero8 --output stabilized/output.mp4 GX010918.MP4
# config files
gyroflow --config stab-low.json --camera hero8 --output stabilized/output.mp4 GX010918.MP4
gyroflow --config stab-high.json --camera hero8 --output stabilized/output.mp4 GX010918.MP4
# multiple files
gyroflow --config stab-low.json --camera hero8 --output stabilized/ GX*.MP4

Hi,
I just wanted to let you know that I have done something similar for myself some time ago.
I wanted to use ReelSteadyGo so I had to fabricate the GPMF track into the mp4 container.

I was successfully able to do this, by converting the blackbox gyro data into GPMF format and RSTG was fooled to accept it.

I was thinking about packaging all up as a project and publish it but didn't have the time and motivation to do it. In my case I was doing it by a bunch of standalone python scripts that require somewhat low lever understanding of how things work.
The main reason I don't feel strong motivation to make it public is because even if it is packaged in nice wizard-like interface there's still considerable manual work to be done which I guess will put off most average users.

Probably the most tricky part is getting the synchronization between the video and the blackbox data right. Of course, this is done with Blackbox explorer, but in my case it seems that the clocks of the video and FC are running at slightly different rate. For example if I sync the video at the beginning of the BB log then at the end it has already drifted. And vice versa - if I sync at the end, then the at the beginning it is incorret. That's why I had to sync at the beginning, write down some numbers, then sync at the end and write down again. Then these both sync points are used in the script to interpolate. It works, of course, but all this is getting quite laborsome for the normal user. Also the syncing in BB explorer must be done quite accurately, which requires some skills too.

In the end run, everything works but it is a hassle.

If you are interested I can share with you my work and you can use it in any way you like. Honestly, I don't know if I'm ever going to work on this on my own to make it public.

Here's an example. It is shot with Runcam 3, integrated the gyro data and run through RSTG. The results are plausible overall. I suspect that it might never be quite perfect, because if the camera is softmounted the microvibrations can quite easily be out of phase with these of the FC. For this reason I doubt that propwash can be properly eliminated but I guess in some cases it will work fine.
https://www.youtube.com/watch?v=CYlGe_PhRLg

Best~

Problem: Most micro and whoop boards lack an onboard blackbox recording option and adding an external Openlager/OpenLogger type device is out of the possible options as most FC's on whoops have 2 or less Uarts and the percentage weight increase is huge in case of micros/whoops.

Proposed workaround: Possibility of using OpenTX telemetry data log as a substitute for a Blackbox Gyro log for video stabilization.

If you recompute the sync it recompute both sync points when you only adjust one sync point. It would be faster two only compute the new sync point

Hello,
In theory it should be possible to estimate the camera angle during calibration by using the gyro data, assuming the monitor is perfectly vertical.

Hi,

I found your repository when trying to find some altenative of ReelsteadyGo for stabilizing fpv video. I was thinking of making one myself by utilising some known stabilization algorithms, beacause I couldn't find any other tool that uses gyroscopic data. However your project seems to be just what I looked for.

I think this project is really promising, and I would love to help you. However in that case a brief explanation of what every script does and what is the structure of the project is necessary...

Are you aware of this repository https://github.com/hovren/crisp ? It is an python library based on this paper https://users.isy.liu.se/en/cvl/perfo/papers/ovren_icra15.pdf The paper focuses mainly on auto-calibration/synchronisation of gyro data and video, which also is what the repository is for. I thought it could be useful in this project.

Btw, are you implementing the stabilisation algorithm from somewhere (e. g. a paper or study) or is it something of your own?

-zap

Can a option be added for the individual axis to turn off stabilize once the gyro rotation goes past a certain speed or acceleration? I'd rather turn off stabilization if my quad clips a branch or taps the ground suddenly.