Auto gyro sync about gyroflow

[ElvinC]

Hey,

I actually haven't seen that paper before. From a quick glance, it can be very useful for this project, since I've been racking my brain trying to find ideas for doing gyro data sync. Thanks for sharing the link!

The overall ideas are based on this paper by Karpenko et al, which also contains a method for gyro sync, but tbh I don't understand the MATLAB code enough to implement it. Ignoring the rolling shutter correction, the paper does image stabilization by having a virtual camera model synced to the raw camera motion. A smooth virtual camera is then computed with a low pass filter, and the difference between the two used to warp the image using a perspective transform. I've been implementing each step independent of the code used for the paper (mostly a bunch of stackoverflow). The paper assumes that the images are taken without distortion (pinhole camera model), so fisheye video has to be undistorted before the stabilization process.

As for what the different files do, here are the important ones.

• Gyroflow.py: Main file with most of the UI code. The UI is made using pyside2. Running this script (with all the dependencies) should show a user interface.
• calibrate_video.py: Contains all the backend code for finding the lens distortion, generating camera presets etc. using OpenCV.
• gyro_integrator.py: Code for integrating gyro data to yield orientation data.
• nonlinear_stretch.py: Mini-project for testing out OpenCV and pyside2.
• perspective_test.py: Testing virtual camera rotation using a perspective transform.
• estimateMotionFromVideo.py: Testing optical flow motion detection in OpenCV (Sorry about the out of place camelCase).

Do take a peek at the code. I've tried to explain most of the stuff with comments and links. Feel free to make a fork if you want to try something out and thanks again for sharing that paper.

Also, you might've noticed that I haven't updated anything the past few weeks. I just moved and started at uni and I'm waiting for my new computer to arrive before diving back in :)

[zap-wizard]

Thanks for the explanation! I will spend some time looking at the code and the paper to understand the main concepts and math behind it first :)

[leewensong]

https://vimeo.com/347404342
I found ReelsteadyGo is not fully automatic, they need sync points to match the gyro timeline. hope this helpful

[ElvinC]

Thanks. The current experimental sync code in Gyroflow uses a similar method where optical flow is computed at two different timestamps and used to compute the offsets. I'm hoping to further fine-tune the sync code for use with blackbox data and finish the basic UI soon (haven't had a lot of spare time until now).