goodrobots / visiond Goto Github PK

Jetson nano CSI port has a special source

zeroconf advertise port for webrtc hardcoded to 6011. Should either be set in config or try and read from janus websocket/http messages.

Currently there is no support for detecting and using video modes - it either takes what is given in the config file, or defaults to 640x480/30. Should detect and verify modes and drop back to a supported default otherwise.
goodrobots/maverick#640

Config file is currently hardcoded to /srv/maverick/config/vision/maverick-visiond.conf.
Add a -c parameter so the path to the config file can be specified, and set the default to the current directory of the visiond script.

visiond should by default startup and autodetect everything it needs to run. the config file should be used to override defaults or to help visiond if it gets stuck autodetecting.

visiond has been migrated to python3, but some warnings remain over legacy code:

./visiond:29: PyGIDeprecationWarning: Since version 3.11, calling threads_init is no longer needed. See: https://wiki.gnome.org/PyGObject/Threading
  GObject.threads_init()
Starting maverick-visiond
./visiond:623: DeprecationWarning: The SafeConfigParser class has been renamed to ConfigParser in Python 3.2. This alias will be removed in future versions. Use ConfigParser directly instead.
  Config = ConfigParser.SafeConfigParser()
./visiond:436: PyGIDeprecationWarning: GObject.MainLoop is deprecated; use GLib.MainLoop instead
  GObject.MainLoop().run()

Currently the config module doesn't have any of the config options defined - it just takes whatever is defined in the config file. But the code refers to various config options throughout, and fails if the options are not present. The config file may not always have (often won't) all of the possible options.

In addition to broadcasting visiond rtsp/rtp streams so clients such as GCS can find them, it should also broadcast janus streams so -discovery can find the webrtc endpoints and -web can pick them up automatically.

Add a name parameter that allows a descriptive name to be set in config. This is useful as it will be passed on in zeroconf and hence -discovery -> -web.

Currently encoder = h264 or autodetected h264 encoding runs through a series of hardware/gstreamer module checks to autodetect the 'best' available encoder, eg. raspberry/jetson hardware encoder.

These encoders don't always work, so it should be possible to choose the encoder (Eg. x264 software encoder).

Currently we have to force certain rtp/rtsp conformity in Janus because the h264 stream from visiond isn't compatible with the browser webrtc support. Figure out what settings are compatible with widest set of browsers and set as default in the encoders.

visiond was traditionally part of Maverick project (https://github.com/goodrobots/maverick) and was known as maverick-visiond. Most of the paths are hardcoded for /srv/maverick paths. Make generic/configurable.

Once a successful connection and stream is made, any subsequent connections fail:

Feb 09 17:04:13 maverick-raspberrylite python3[15446]: 2020-02-09 17:04:13,428 - Visiond - INFO - Creating RTSP factory element: <GstRtsp.RTSPUrl object at 0xb5520ab0 (GstRTSPUrl at 0x2105ad0)>
Feb 09 17:04:13 maverick-raspberrylite python3[15446]: gst_ghost_pad_new: assertion '!gst_pad_is_linked (target)' failed
Feb 09 17:04:13 maverick-raspberrylite python3[15446]: gst_pad_set_active: assertion 'GST_IS_PAD (pad)' failed
Feb 09 17:04:13 maverick-raspberrylite python3[15446]: gst_element_add_pad: assertion 'GST_IS_PAD (pad)' failed
Feb 09 17:04:13 maverick-raspberrylite python3[15446]: gst_rtsp_stream_new: assertion 'GST_IS_PAD (pad)' failed
Feb 09 17:04:13 maverick-raspberrylite python3[15446]: gst_rtsp_stream_set_multicast_iface: assertion 'GST_IS_RTSP_STREAM (stream)' failed
Feb 09 17:04:13 maverick-raspberrylite python3[15446]: gst_rtsp_stream_set_profiles: assertion 'GST_IS_RTSP_STREAM (stream)' failed
Feb 09 17:04:13 maverick-raspberrylite python3[15446]: gst_rtsp_stream_set_protocols: assertion 'GST_IS_RTSP_STREAM (stream)' failed
Feb 09 17:04:13 maverick-raspberrylite systemd[1]: maverick-visiond.service: Main process exited, code=killed, status=11/SEGV
Feb 09 17:04:13 maverick-raspberrylite systemd[1]: maverick-visiond.service: Failed with result 'signal'.

Testing last night on the webrtc-dev branch
default settings
pi3b+ with picam

rtsp pipeline was created OK
rtsp pipeline fails on connection

Logs to come tonight

@fnoop Throwing this out there as an idea for comments. For maverick do we want visiond to be the one stop shop for all camera interfaces? E.g. when we start looking at supporting automated DSLR cameras or other machine vision cameras (point grey, ximea, etc) do you envision visiond doing this or a separate service?

visiond should broadcast available streams, so other clients can pick it up such as -api. I think camera-manager also does something similar - look at what they're broadcasting as they might have an established format that GCS will pick up.

Look at possible nvidia support from here:
#6

webrtc service has to be restarted for visiond to pick up janus and broadcast zeroconf.

Also an error:

Apr 16 17:19:59 maverick-raspberrylite python3[686]: 2020-04-16 17:19:59,124 - zeroconf - WARNING - Exception occurred:
Apr 16 17:19:59 maverick-raspberrylite python3[686]: Traceback (most recent call last):
Apr 16 17:19:59 maverick-raspberrylite python3[686]:   File "/srv/maverick/software/python/lib/python3.7/site-packages/zeroconf/__init__.py", line 2638, in send
Apr 16 17:19:59 maverick-raspberrylite python3[686]:     bytes_sent = s.sendto(packet, 0, (real_addr, port))
Apr 16 17:19:59 maverick-raspberrylite python3[686]: OSError: [Errno 101] Network is unreachable

Encoder Bitrate is currently not set or hardcoded. Add parameter to allow this to be set in config.

v4l2 is default, but should be auto-detect

Logging is currently output to file. Filepaths are set to /srv/maverick and need to be reset relative to script, or else change logging to output to stdout/stderr, or systemd/journald.

Add parameters to set horizontal/vertical flip

Testing last night on the webrtc-dev branch
pi3b+ with picam

Service fails to exit. Bug likely introduced during webrtc-dev work.