vladimirvivien / go4vl

A Go library for working with the Video for Linux API (V4L2).

go4vl

A Go centric abstraction of the library for Video for Linux 2 (v4l2) user API.

---

The go4vl project is for working with the Video for Linux 2 API for real-time video. It hides all the complexities of working with V4L2 and provides idiomatic Go types, like channels, to consume and process captured video frames.

This project is designed to work with Linux and the Linux Video API only. It is NOT meant to be a portable/cross-platform package.

Features

• Capture and control video data from your Go programs
• Idiomatic Go types such as channels to access and stream video data
• Exposes device enumeration and information
• Provides device capture control
• Access to video format information
• Streaming users zero-copy IO using memory mapped buffers

Compilation Requirements

• Go compiler/tools
• Kernel minimum v5.10.x
• A locally configured C compiler (or a cross-compiler if building off-device)

See example/README.md for further example of how to build projects that uses go4vl, including cross-compilation.

All examples have been tested using a Raspberry PI 3, running 32-bit Raspberry PI OS. The package should work with no problem on your 64-bit Linux OS.

Getting started

Using the go4vl package

To include go4vl in your own code, go get the package:

go get github.com/vladimirvivien/go4vl/v4l2

Video capture example

The following is a simple example that shows how to capture a single frame from an attached camera device and save the image to a file.

The example assumes the attached device supports JPEG (MJPEG) output format inherently.

func main() {
	dev, err := device.Open("/dev/video0", device.WithBufferSize(1))
	if err != nil {
		log.Fatal(err)
	}
	defer dev.Close()

	if err := dev.Start(context.TODO()); err != nil {
		log.Fatal(err)
	}

	// capture frame
	frame := <-dev.GetOutput()

	file, err := os.Create("pic.jpg")
	if err != nil {
		log.Fatal(err)
	}
	defer file.Close()

	if _, err := file.Write(frame); err != nil {
		log.Fatal(err)
	}
}

See complete example here.

Examples

This repository comes with several examples that show how to use the API to build Go programs that can capture images from Linux.

See list of examples

Roadmap

The main goal is to port as many functionalities as possible so that adopters can use Go to create cool video-based tools on platforms such as the Raspberry Pi.

go4vl's Issues

Is go4vl Tinygo compiler compatible?

It would be great to spend time and validate whether go4vl APIs can be compiled with the tiny go compiler.

See https://tinygo.org/docs/guides/linux/

Single capture at regular intervals

What's the best approach to do a single capture at regular intervals.
There is an example for snapshot, but that one closes the device once it finishes storing the image.

In a scenario where we want to regularly capture, sometimes with a couple seconds difference, others with minutes, should the device be closed or kept open?

If open, what is the best approach to ensure getting the latest frame without causing any issues or large memory footprint?

Thanks

resource busy error while device.open()

Hi all,
while using the code on this link , https://github.com/vladimirvivien/go4vl/blob/main/examples/capture0/capture0.go , I am facing resource busy error after some iterations at the instance where it tries to open the device. I am using the exact same code but in loop.

please let me know for any suggestions.
Thanks in advance.

Format and FrameSize Enum

Add code to enumerate supported device format and fame size.
See vidioc_enum_fmt and VIDIOC_ENUM_FRAMESIZES

Incorrect parsing of IntegerMenu's menu items with string type

reproduce

When I use the following code to print Control, there are garbled characters.

https://github.com/vladimirvivien/go4vl/blob/018089c752cb092417d259661d6d8dc3874fc319/examples/ext_ctrls/extctrls.go#LL37C1-L52C2

expected behavior

prints the value correctly like the v4l2-ctl -L

possible reason

The program treats menu items as strings, but actually when the menu type is IntegerMenu, the value should be treated as Int64.
Refer to this document description.

go4vl/v4l2/control.go

Lines 176 to 182 in 018089c

	func makeCtrlMenu(qryMenu C.struct_v4l2_querymenu) ControlMenuItem {
	return ControlMenuItem{
	ID: uint32(qryMenu.id),
	Index: uint32(qryMenu.index),
	Name: C.GoString((*C.char)(unsafe.Pointer(&qryMenu.anon0[0]))),
	}
	}

version

• go4vl: v0.0.5
• golang: 1.19

https://github.com/vladimirvivien/go4vl/blob/018089c752cb092417d259661d6d8dc3874fc319/examples/ext_ctrls/extctrls.go#LL37C1-L52C2

default fps when capture

nice job .
i see the default/base fps is set to 10 from here :

go4vl/v4l2/device/device.go

Line 231 in 40b41ba

fps = 10

, is there any way to detect and set a default fps value(rather than 10) supported by opened device ?
thanks

Type clean up and consistency

• Make type export consistent
• Clean up code
• Apply consistent naming

capture video and audio at same time possible ?

I wanna capture both video and audio from a HDMI to USB Capture Card , and keep video and audio in sync , is that possible using this library ? if possible , could you show me an example ?
Thanks

enumerate the devices from /sys/class/video4linux instead of /dev

to simplify https://github.com/vladimirvivien/go4vl/blob/v0.0.5/device/list.go it would be simpler to enumerate the files/symlinks at /sys/class/video4linux. or maybe there was a good reason for directly digging in /dev?

Stream TO device?

Let's say I have a network stream that outputs an mpegts stream. How can I pipe that into a v4l2 device?

Source code unit testing

Use a fake v4l2 device to implement source code unit tests.

Non-blocking device file may be causing invalid captured buffer

Currently, the a device is opened as non-blocking and the code uses select to be notified of buffer readiness. This appears to cause incomplete or incorrect buffers to be returned. This could be due to the driver still writing while buffer is being read. The current workaround has been to copy the buffer before making it available for use.

Explore other approaches including blocking read, DMA, or direct device reads or make them all available at some point and let developer select which work best for device being tested.

Reduce cpu usage

Hello,

i'm planing on using three webcams on a raspberry pi, but noticed that even when reducing the framerate to "1", the cpu usage is very high (>100%) even for only one webcam.

Is there anyway to reduce the cpu usage? Thanks in advance.

Kevin

Exploring ways to integrate/automate C header files in porject

In #20, @ajcasagrande contributed code changes that copied the Kernel v4l2 header files from a specific commit. This provides for a more stable build experience because the code does not depend on the local machine to provide the header files. This is very useful, however @farshidtz pointed out, in the closed PR, that there could be some unforeseen additional issues.

I am opening this issue provide a place to continue discussions on this and also propose additional ways of approaching this problem. There are some valid issues brought forth in the discussion above. I am doing some research to see how other Go projects handle this. What I have seen so far

• Project that includes the headers, as is done here
• Project that automatically pull a specific version at commit, via submodules

I think there is a middle ground solutions that can be adopted, while avoiding any additional burden for users of this library.

Possible solutions

• Manually update header files and pinned to a specific version (as done now)
• Creating a simple code gen (Makefile/script) to automates pulling header files for a specified Kernel version.
• Will add more proposals as I continue to explore

Use types in Go's image package

The Go's image package has some surprisingly great type overlaps with this project.
It would be great to leverage these types in the future

• Bound
• Rect
• YCbCr
• RGB
• Gray
• JPG
• MPEG
• Etc

DMA-BUF support

I want to preview captured frame as fast/performant as possible , i.e. pass dma buf to gstreamer／drm to display it (zero-copy) , could this lib export such an option (DMA BUF ) in addition to MMAP way?

Thanks

Error run or build

Raspberry Pi 4
Go 1.21
CMD: env GOARCH=arm64 GOOS=linux go run cmd/main.go
Output:

# github.com/vladimirvivien/go4vl/v4l2
../go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/types.go:11:15: undefined: Capability
../go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/types.go:12:14: undefined: IOType
../go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/types.go:23:15: undefined: BufType

Add support for YUYV

Currently only MJPEG is supported but it would be great if there was support for the YUYV format.
Showing:

2021/07/31 15:35:43 setting format to 640x480 YUYV
2021/07/31 15:35:43 device or resource busy
exit status 1

Sample camera settings:

	Driver name      : ov534
	Card type        : USB Camera-B4.09.24.1
	Bus info         : usb-3f980000.usb-1.5
	Driver version   : 5.10.17
	Capabilities     : 0x85200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps      : 0x05200001
		Video Capture
		Read/Write
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (ov534: ok)
Format Video Capture:
	Width/Height      : 640/480
	Pixel Format      : 'YUYV' (YUYV 4:2:2)
	Field             : None
	Bytes per Line    : 1280
	Size Image        : 614400
	Colorspace        : sRGB
	Transfer Function : Default (maps to sRGB)
	YCbCr/HSV Encoding: Default (maps to ITU-R 601)
	Quantization      : Default (maps to Limited Range)
	Flags             :
Streaming Parameters Video Capture:
	Capabilities     : timeperframe
	Frames per second: 30.000 (30/1)
	Read buffers     : 2````

New version release?

Would you consider releasing a new version? I am currently using several of the camera controls that are not in 0.0.5, but have been added in main.

Golang 1.20 compiler errors

There appear to be some changes in 1.20 that cause the library to not compile and report some issues on my windows machine.
Reverting to the latest 1.19 makes it work again.

It appears to be related to the use of consts and 1.20 doesn't seem to like it for some reason.
Found some cases with a similar report that implied a constant overflow check, but your code seems correct in that regard.

It reports this with go build ./... on a project:

........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\types.go:11:15: undefined: Capability
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\types.go:12:14: undefined: IOType
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\types.go:23:15: undefined: BufType
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\syscalls.go:33:17: undefined: sys.Openat
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\syscalls.go:33:28: undefined: sys.AT_FDCWD
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\syscalls.go:53:13: undefined: sys.Close
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\syscalls.go:57:43: undefined: sys.Errno
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\syscalls.go:59:22: undefined: sys.Syscall
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\syscalls.go:59:34: undefined: sys.SYS_IOCTL
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\syscalls.go:63:12: undefined: sys.EINTR
........\go\pkg\mod\github.com\vladimirvivien\[email protected]\v4l2\syscalls.go:63:12: too many errors

It also complains inside VSCode anywhere the consts are used within a project.
I don't have the v4l2 source files in my windows box, but 1.19 never complained about it.

Unsure how this could be fixed, if at all, and if this is a golang issue that could be to raise with the go team.
Thought I'd raise it and get your opinion on it.

v4l2/capability.go:48:33: could not determine kind of name for C.V4L2_CAP_IO_MC

uname -r
5.11.0-1022-aws

sudo find / -name videodev2.h
/usr/include/linux/videodev2.h

go version
go version go1.17.8 linux/amd64

GCC

gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Thread model: posix
gcc version 9.4.0 (Ubuntu 9.4.0-1ubuntu1~20.04)

 make
CGO_ENABLED=1 GO111MODULE=on go build -ldflags "-X github.com/edgexfoundry/device-usb-camera.Version=0.0.0" -o cmd/device-usb-camera ./cmd
# github.com/vladimirvivien/go4vl/v4l2
../go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/capability.go:48:33: could not determine kind of name for C.V4L2_CAP_IO_MC
make: *** [Makefile:21: cmd/device-usb-camera] Error 2

not all code paths in device.Open close the underline device on error

Not all the code paths call v4l2.CloseDevice when there is an error at:

https://github.com/vladimirvivien/go4vl/blob/v0.0.5/device/device.go#L28-L116

All the return statements that return an error should call v4l2.CloseDevice.

write to mp4 x264

Great work!! Would be great to see an example where you write and encode to an mp4 using x264.

Add README and documentation

Update readme.md and add documentation

• Add v4l2 info (from blog)
• Explain example (how to setup and run)
• Add TODO.md or Roadmap.

Missing constants from linux/videodev2.h

Hi, I'm getting the following errors building the package,

/home/dev/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/capability.go:48:33: could not determine kind of name for C.V4L2_CAP_IO_MC
/home/dev/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/format_desc.go:25:55: could not determine kind of name for C.V4L2_FMT_FLAG_CSC_COLORSPACE
/home/dev/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/format_desc.go:28:55: could not determine kind of name for C.V4L2_FMT_FLAG_CSC_HSV_ENC
/home/dev/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/format_desc.go:29:55: could not determine kind of name for C.V4L2_FMT_FLAG_CSC_QUANTIZATION
/home/dev/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/format_desc.go:26:55: could not determine kind of name for C.V4L2_FMT_FLAG_CSC_XFER_FUNC
/home/dev/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/format_desc.go:27:55: could not determine kind of name for C.V4L2_FMT_FLAG_CSC_YCBCR_ENC
/home/dev/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/format_desc.go:24:55: could not determine kind of name for C.V4L2_FMT_FLAG_ENC_CAP_FRAME_INTERVAL

When I open /usr/include/linux/videodev2.h I see these constants are not present. This is on ubuntu 20.04.3 with package linux-libc-dev version 5.4.0-94.106 (after update and upgrade). There may be other such constants, it stops showing errors at a certain point.

On my other popos install I see linux-libc-dev is at version 5.15.5 and looks like the constants are present.

It looks like some of those constants are relatively new, like V4L2_CAP_IO_MC might just be from 2020.

Do I need to find a way to get the newer package? or should this library try to accommodate? Thanks

Capture errors when webcam is removed

Is there any way to catch errors when the user removes the webcam while in usage?
In the simplecam example, this panic comes up when i remove the webcam while in use:

2023/03/03 23:24:50 Serving images: [:9090/stream]
panic: device: stream loop dequeue: buffer dequeue: system error

goroutine 18 [running]:
github.com/vladimirvivien/go4vl/device.(*Device).startStreamLoop.func1()
        /home/kk/go/pkg/mod/github.com/vladimirvivien/[email protected]/device/device.go:409 +0x4ac
created by github.com/vladimirvivien/go4vl/device.(*Device).startStreamLoop
        /home/kk/go/pkg/mod/github.com/vladimirvivien/[email protected]/device/device.go:392 +0x1ec
exit status 2

Simplecam example not running on Rasberry pi 4 with camera module 3

Hi there,

Trying to run the simplecam example on a raspberry pi 4 with camera module 3 and I get the following error:

camera start: device: start stream loop: device: stream on: stream on: bad argument error

I did follow the step mentioned here and I am building the binaries from the PI itself
https://medium.com/go4vl/build-a-wifi-camera-using-the-raspberry-pi-zero-w-a-camera-module-and-go-1d5fadfa7d76

Seem like a bug but feel free to close if it is not supported on this hardware.

Using go 1.20 and Raspberry os

Thanks in advance for any help!

Potential memory leak

While testing the webcam example on a raspberry pi, top shows that the memory continues to grow.

Is anyone else seeing this behavior?

Cannot retrieve pixel format info from device [Error While Encoding v4l2 command via ioctl]

Error

device.GetPixFormat() always fails with error device: pix format failed: unsupported error
if we do a system call trace i get following

ioctl(3, _IOC(_IOC_READ|_IOC_WRITE, 0x56, 0x4, 0xcc), 0xc000076ca8) = -1 ENOTTY (Inappropriate ioctl for device)
write(1, "failed to get pix format\n", 25failed to get pix format

It looks like encoded command is not VIDIOC_G_FMT and is not recognized by kernel.

package main

import (
	"fmt"

	"github.com/vladimirvivien/go4vl/v4l2"
)

func main() {
	device, err := v4l2.Open("/dev/video0")
	if err != nil {
		panic(err)
	}

	_, err = device.GetPixFormat()
	if err != nil {
		fmt.Println("failed to get pix format")
		panic(err)
	}
}

My kernel is

Linux endless 5.14.14-arch1-1 #1 SMP PREEMPT Wed, 20 Oct 2021 21:35:18 +0000 x86_64 GNU/Linux

Cause

I don't think this problem is caused by kernel.
I can use <linux/videodev2.h> and it does not cause this error

With the C API I get

ioctl(3, VIDIOC_G_FMT, {type=V4L2_BUF_TYPE_VIDEO_CAPTURE, fmt.pix={width=1280, height=720, pixelformat=v4l2_fourcc('M', 'J', 'P', 'G') /* V4L2_PIX_FMT_MJPEG */, field=V4L2_FIELD_NONE, bytesperline=0, sizeimage=1843200, colorspace=V4L2_COLORSPACE_SRGB}}) = 0

It looks like the problem is caused in ioctl encoding functions. I rechecked the ioct.go but could not find the problem

Buffer dequeue error

Hi,

I'm testing out the mjpeg saving example in the README (making the sole change of width to 1280 and height to 720 - which the camera DOES support) and half of the time the example runs smoothly and then sometimes when I run it I get the following error:

Done.
panic: device: capture: buffer dequeue: system error

goroutine 34 [running]:
github.com/vladimirvivien/go4vl/v4l2.(*Device).Capture.func1(0xd06000, 0x3, 0xcb0850, {0x104658, 0xd02000}, 0x1fca055)
	/home/pi/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/device.go:245 +0x3c0
created by github.com/vladimirvivien/go4vl/v4l2.(*Device).Capture
	/home/pi/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/device.go:230 +0xd0
exit status 2

Have you seen that before or have any idea why it might be happening seemingly randomly?

Thanks for the library btw!

possible add write functionality to a v4l2 device ?

is it possible to make this library could write to a v4l2 device i.e. to a virtual camera ?

How to build the "webcam" example for raspi os (32bit) on linux amd64 machine?

When I try to build the example in go4vl/examples/webcam directory on my machine (linux amd64) for ARM architecture I get following error:

> GOARCH=arm ./build.sh 
# github.com/vladimirvivien/go4vl/v4l2
/home/me/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/types.go:11:15: undefined: Capability
/home/me/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/types.go:12:14: undefined: IOType
/home/me/go/pkg/mod/github.com/vladimirvivien/[email protected]/v4l2/types.go:23:15: undefined: BufType

What I am doing wrong?
What should I do?

add timestamp choice and conversion methods ?

i see from ffmpeg v4l2 , it has :

/**

• Return timestamps to the user exactly as returned by the kernel
  /
  #define V4L_TS_DEFAULT 0
  /*
• Autodetect the kind of timestamps returned by the kernel and convert to
• absolute (wall clock) timestamps.
  /
  #define V4L_TS_ABS 1
  /*
• Assume kernel timestamps are from the monotonic clock and convert to
• absolute timestamps.
  */
  #define V4L_TS_MONO2ABS 2

is it implemented in this project , i think it might be a useful feature .

P.S. this project evolves a lot since my last visit , great job .

Are there plans to provide a utility to convert YUYV format to go image.Image?

Now, we can get the source data in YUV format, but there is no corresponding tool to convert it to the image format supported by go.
Is there a support plan?

Generating camera image

The examples seem to focus on capturing camera images.

Would this library also be fit to "generate" camera input, to be consumed by e.g. OBS Studio or Zoom? I've been able to output OBS Studio to v4l2 to be consumed by Zoom, so I'd assume v4l2 in general somehow supports it.

The IsVideoOutputSupported function seems to indicate something like that is possible.

Are there any pointers you can give, where one would get started on doing so?

func writeFrame(frame *image.RGBA) {
    // My custom logic to write this frame
}

func main() {
	devName := "/dev/video10"
	device, err := v4l2.Open(devName)
	if err != nil {
		log.Fatalln("unable to open virtual webcam:", err)
	}

	defer device.Close()

	caps, err := device.GetCapability()
	if err != nil {
		log.Println("failed to get device capabilities:", err)
	}
	log.Printf("device [%s] opened\n", devName)
	log.Printf("device info: %s", caps.String())

	if !caps.IsVideoOutputSupported() {
		log.Fatalln("virtual webcam does not support video output")
	}

	// Something should go here, to prepare everything for "write mode"
	pxFmt, err := GetPixFormatOutput(device.fd)
	if err != nil {
		log.Fatalln("unable to get pix format:", err) // It crashes here due to https://github.com/vladimirvivien/go4vl/issues/7 but that's OK
	}

	// And at some point, whenever a frame should be written, the `writeFrame` above can be called, and those bytes could
	// then be written into the buffer allocated for v4l2
}

func GetPixFormatOutput(fd uintptr) (PixFormat, error) {
	format := v4l2Format{StreamType: BufTypeVideoOutput}
	if err := Send(fd, VidiocGetFormat, uintptr(unsafe.Pointer(&format))); err != nil {
		return PixFormat{}, fmt.Errorf("pix format failed: %w", err)
	}

	return format.getPixFormat(), nil
}

I'm using the code for obs-v4l2sink as a guidance, because I know that code works.

Unable to get RAW image from usb camera

Hi @vladimirvivien
I am using /examples/capture0 folder to capture the image from usb camera in a raw image format(as . raw file) by changing the pixel format to PixelFormat: v4l2.PixelFmtRGB24 and saving the file as .raw extension. but i am not able to get valid raw image.
so is there any necessary to change in
Colorspace : sRGB
YCbCr/HSV encoding : Default (map to ITU-R 601)
Quantization : Default (map to Full range)
Supported formats: YUyv and Motion-JPEG

Review/refactor datapath to avoid unnecessary allocations and resource leakage

Now that the control portion of the API seems to be stable. It is time to go back into the datapath API to do some remodeling. This started with #22 . It appears that some early decisions in that portion of the code needs some refactoring to ensure better stability and avoid resource leakage.

Unable to set any frame size other than the default 640x480

Hi, I seem to have a problem setting the image size correctly and it keeps defaulting to the default 640x480, even when using your webcam.go example and changing the with and height to 1280x720 still resolves in a 640x480 picture. the capture card is able to provide an MJPEG of 720p as well as 1080p and this also works via this pkg: https://github.com/blackjack/webcam
but I cant get the same to work here, even though I like the rest of the implementation here a bit better.

Would you mind giving an example of a working webcam.go or capture.go with a different image size just as a reference to look at.

`v4l2.CtrlRotate` errors

I'm trying to use v4l2.CtrlRotate to get/set frame rotation, but I get the following errors, depending on whether I try to get or set, respectively:

# cam.GetControl(v4l2.CtrlRotate)
device: /dev/video0: get control: query control info: VIDIOC_QUERYCTRL: id 9963810: bad argument error

# cam.SetControlValue(v4l2.CtrlRotate, 180)
device: /dev/video0: set control value: id %!s(uint32=9963810): query control info: VIDIOC_QUERYCTRL: id 9963810: bad argument error

I don't have an MVP right now (I can get one if you think it worth it), but the operations are roughly these, in this order (errors are ignored only for this excerpt):

cam, err := device.Open(dev, device.WithBufferSize(uint32(NWORKERS-1)))

rot, err := cam.GetControl(v4l2.CtrlRotate)      // <---
err := cam.SetControlValue(v4l2.CtrlRotate, 180) // <---

err := cam.SetPixFormat(v4l2.PixFormat{
	// Chosen by a controler program, from the set of possible values
	Width:        fmts[wi].Width,
	Height:       fmts[wi].Height,
	PixelFormat:  fmts[wi].pixelFormat,

	// Basically defaults, for reasons...
	Field:        v4l2.FieldNone,
	Colorspace:   v4l2.ColorspaceSRGB,
	YcbcrEnc:     v4l2.YCbCrEncoding601,
	Quantization: v4l2.QuantizationDefault,
	XferFunc:     v4l2.XferFuncSRGB,
})
err := cam.SetFrameRate(fps) // Also chosen by the controller program
err := cam.Start(ctx)
frames := cam.GetOutput()
for frame := range frames {
	// ...
}

The program works really well without the two CtrlRotate lines; the error happens only when trying to use CtrlRotate. The errors show up whether I put the CtrlRotate lines before or after SetPixFormat. Tried this because of the message in the V4L2 docs:

It is necessary to set the new height and width of the picture using the VIDIOC_S_FMT ioctl according to the rotation angle selected

Given the error message I suspect the CtrlID is wrong? I saw it's defined in go4vl itself. I saw this page linked from the docs, too; maybe it's just my computer? I couldn't find the relevant V4L2 header(s) or V4L2_CID_ROTATE on my computer to confirm this, however...

Am I using these functions incorrectly? Could it be kernel related? Camera related? Any ideas/help is appreciated!

Incorrect parsed FrameSize in getFrameSize function

reproduce

Use the following code to view the supported Frame Sizes.

func main() {
	devName := "/dev/video0"
	dev, err := device.Open(
		devName,
	)
	if err != nil {
		panic(err)
	}
	info, err := v4l2.GetAllFormatFrameSizes(dev.Fd())
	if err != nil {
		panic(err)
	}
	marshal, err := json.MarshalIndent(info, "", "    ")
	if err != nil {
		panic(err)
	}
	fmt.Println(string(marshal))
}

The following error value was returned.

[
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 842093913,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 1448695129,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 859981650,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 1195724874,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 875967048,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 1196444237,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 1431918169,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 1498765654,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 1498831189,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 842094158,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 861030210,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 842094169,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 825382478,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    },
    {
        "Index": 0,
        "Type": 3,
        "PixelFormat": 875714642,
        "Size": {
            "MinWidth": 2,
            "MaxWidth": 32,
            "StepWidth": 2464,
            "MinHeight": 2,
            "MaxHeight": 0,
            "StepHeight": 0
        }
    }
]

expected behavior

Returned the correct value.

{
    "Index": 0,
    "Type": 3,
    "PixelFormat": 875714642,
    "Size": {
        "MinWidth": 32,
        "MaxWidth": 3280,
        "StepWidth": 2,
        "MinHeight": 32,
        "MaxHeight": 2464,
        "StepHeight": 2
    }
}

possible reason

From the output, we can see that the value has been shifted forward by 2 units. The guess is that there is an error in the type conversion from c to golang.

go4vl/v4l2/format_framesizes.go

Lines 70 to 72 in efb35e7

	case FrameSizeTypeStepwise, FrameSizeTypeContinuous:
	// Calculate pointer to access stepwise member
	frameSize.Size = *(*FrameSize)(unsafe.Pointer(uintptr(unsafe.Pointer(&frmSizeEnum.anon0[0])) + unsafe.Sizeof(FrameSizeDiscrete{})))

According to the documentation, size information is contained in a union. The problem was fixed using the following code.

	case FrameSizeTypeStepwise, FrameSizeTypeContinuous:
		// Calculate pointer to access stepwise member
		frameSize.Size = *(*FrameSize)(unsafe.Pointer(&frmSizeEnum.anon0[0]))

Implement device control

Add code to allow device control using v4l2_ext_control/v4l2_ext_controls.

Build fails in dockerfile

Attempting to build in docker file and receiving the following error:

$ docker build --platform "linux/arm/v6" --output "./build" .
[+] Building 18.3s (12/13)
 => [internal] load build definition from Dockerfile                                                                                                                                                        0.0s
 => => transferring dockerfile: 1.52kB                                                                                                                                                                      0.0s 
 => [internal] load .dockerignore                                                                                                                                                                           0.0s 
 => => transferring context: 2B                                                                                                                                                                             0.0s 
 => resolve image config for docker.io/docker/dockerfile:1                                                                                                                                                  0.3s 
 => CACHED docker-image://docker.io/docker/dockerfile:1@sha256:39b85bbfa7536a5feceb7372a0817649ecb2724562a38360f4d6a7782a409b14                                                                             0.0s
 => [internal] load .dockerignore                                                                                                                                                                           0.0s
 => [internal] load build definition from Dockerfile                                                                                                                                                        0.0s 
 => [internal] load metadata for docker.io/crazymax/goxx:latest                                                                                                                                             0.3s 
 => [internal] load build context                                                                                                                                                                           0.2s
 => => transferring context: 50.60kB                                                                                                                                                                        0.1s 
 => [base 1/2] FROM docker.io/crazymax/goxx:latest@sha256:e39c04878d7249622367528c86d4cbedac3a8c916a2177400f055edfc36f6b51                                                                                  0.0s 
 => CACHED [base 2/2] WORKDIR /out                                                                                                                                                                          0.0s 
 => [build 1/2] RUN --mount=type=cache,sharing=private,target=/var/cache/apt   --mount=type=cache,sharing=private,target=/var/lib/apt/lists   goxx-apt-get install -y binutils gcc g++ pkg-config          16.8s 
 => ERROR [build 2/2] RUN --mount=type=bind,source=.   --mount=type=cache,target=/root/.cache   --mount=type=cache,target=/go/pkg/mod   goxx-go build -o /out/simple-cam .                                  0.7s
------
 > [build 2/2] RUN --mount=type=bind,source=.   --mount=type=cache,target=/root/.cache   --mount=type=cache,target=/go/pkg/mod   goxx-go build -o /out/simple-cam .:
#12 0.700 # github.com/vladimirvivien/go4vl/v4l2
#12 0.700 vendor/github.com/vladimirvivien/go4vl/v4l2/capability.go:51:33: could not determine kind of name for C.V4L2_CAP_IO_MC
------
executor failed running [/bin/sh -c goxx-go build -o /out/${OUTPUT} .]: exit code: 1

Using the provided dockerfile in the simple webcam example. Same code compiles and runs successfully on a Pi 4.

Add Go test functions

The code has little test functions included in the source code. It should be possible to use the mock Loopback V4L2 device to create tests that exercise more of the current code. This could be coupled with a containerized environment to allow the code to be reproduced on any platform.