Comments (7)
Yep, that looks about correct (given that the frames you get from the camera are indeed in the YUYV
order).
from libv4l-rs.
Well this library is format agnostic. If you are asking whether ffimage
will be able to convert those formats into packed RGB pixels, the answer is probably no. It should not be hard to add the conversion routines for those formats though. The existing YUV <-> RGB code should give you an idea of how to convert the image data.
There's even existing code handling YUV420p: https://github.com/raymanfx/ffimage/blob/master/ffimage-yuv/src/yuv420.rs. So if you want RGB pixels, you'll need to implement the YU12 (planar) -> YUV444 (packed) conversion step and the complete pipeline would probably look like this: YU12 -> YUV444 -> RGB
.
from libv4l-rs.
So stream.next()
, e.g. for the memory-mapped IO transport (see: https://github.com/raymanfx/libv4l-rs/blob/master/examples/stream_capture_mmap.rs#L35) gives you a buffer which is essentially a slice of bytes: &[u8]
.
It is important that you configure the v4l2 device to use a format that is supported by image-rs, e.g. RGB888. Most webcams will not support that, but only provide YUYV (aka Yuv422). I saw your comment on the image-rs repo and unfortunately it's still not something they support (AFAIK). You can use ffimage v0.10.0
from crates.io to convert a slice of YUYV bytes to a slice of RGB bytes (see: https://github.com/raymanfx/ffimage/blob/next/ffimage-yuv/benches/convert.rs#L29).
You should be able to then use ImageBuffer::from_raw()
from image-rs (see: https://docs.rs/image/latest/image/struct.ImageBuffer.html#method.from_raw) and pass it those bytes.
Let me know if this helps; and sorry for the late reply!
from libv4l-rs.
Great question, I would really like to use this library but I too am struggling with the capture buffer conversion to an image format I can work with. The ffimage example is not clear to me .. any further guidance would be greatly appreciated.
from libv4l-rs.
I think I have figured it out:
use image::{ImageBuffer};
use v4l::buffer::Type;
use v4l::io::traits::CaptureStream;
use v4l::prelude::*;
use v4l::video::Capture;
use ffimage_yuv::yuv422::Yuv422;
use ffimage_yuv::yuv::Yuv;
use ffimage::color::Rgb;
use ffimage::iter::{BytesExt, ColorConvertExt, PixelsExt};
// in your main function (or whatever).
let mut dev = Device::new(0).expect("Failed to open device");
let format = dev.format()?;
let (buf, meta) = stream.next().unwrap();
// unwinding the expression below in comments:
let rgb: Vec<u8> = buf.iter()
.copied() // Vec<u8>
.pixels::<Yuv422<u8, 0, 2, 1, 3>>() // Vec<Yuv422<u8, 0, 2, 1, 3>> // after pixels
.colorconvert::<[Yuv<u8>; 2]>() // Vec<[Yuv<u8>;2]> // after colorconvert::<[Yuv<u8>; 2]>()
.flatten() // Vec<Yuv<u8>> // after flatten
.colorconvert::<Rgb<u8>>() // Vec<Rgb<u8>> // after colorconvert::<Rgb<u8>>
.bytes() // Vec<[u8;3]> // after bytes;
.flatten()
.collect();
// https://github.com/raymanfx/libv4l-rs/issues/74
// NOTE: dont confuse image::Rgb with ffimage::Rgb
let img: ImageBuffer<image::Rgb<u8>, Vec<u8>> = ImageBuffer::from_raw(
format.width,
format.height,
rgb,
)?;
img.save("test.png");
from libv4l-rs.
@raymanfx many thanks. This works for one of my cameras but my other camerahas a format that is Y16 (grayscale)? Seems it also supports Yu12 (YUV4:2:0) and NV12 (Y/CbCr 4:2:0) will your library work with these formats? Sorry for the dumb questions but this rather new to me.
from libv4l-rs.
Brilliant thank you @raymanfx . I was able to get it working wit the following snippet (replacing the critical section in the code above)
let rgb: Vec<u8> = Yuv420p::pack(&buf, 640, 512)
.into_iter()
.colorconvert::<Rgb<u8>>()
.bytes()
.flatten()
.collect();
The images look Ok, but this is a grayscale (IR camera) and I suspect this is not efficient. I'll study your suggestion and see what I can figure out.
from libv4l-rs.
Related Issues (20)
- new release HOT 4
- Missing reset_controls HOT 1
- Problems I encountered using this crate. HOT 4
- Failed to start media pipeline HOT 2
- Accessing to Handle for Streams HOT 1
- Publish `next` to crates.io HOT 2
- Hardware Encoding/Decoding HOT 1
- please release a new version of v4l2-sys-mit HOT 1
- Unable to build for RISC-V HOT 2
- Any way to tell device to write into a memory region i defined? HOT 1
- Publish `Stream::handle()` on crates.io HOT 1
- Crash on mem::zeroed HOT 3
- Getting zeroes in capture buffers HOT 3
- Manually starting a CaptureStream results in a single queued buffer HOT 1
- Build fail when features=libv4l HOT 2
- Unhelpful error message when compiling with default features off HOT 1
- Fails to build on Fedora "v4l2_pix_format_union_(anonymous_at_/usr/include/linux/videodev2_h_500_2)" is not a valid Ident HOT 3
- OS Invalid input error HOT 2
- Panic: v4l-0.14.0/src/control.rs:172:45: called Result::unwrap() on an Err value: () HOT 15
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from libv4l-rs.