Comments (9)
#72 (comment)
Source frame format NV12 may not be supported on macintosh untill now, I guess.
from nokhwa.
It is likely a fault of the library improperly setting FourCC
from nokhwa.
I found that the first problem is the NV12 checker in this line:
nokhwa/nokhwa-core/src/types.rs
Line 1445 in ad93e0a
if data.len() != ((resolution.width() * resolution.height() * 3) / 2) as usize {
I tried to change it into
if data.len() != (resolution.width() * resolution.height() * 2) as usize {
Now it passes the decoding round.
But the resulting decoded image is faulty. It looks like it's split in the middle, and then we put the left part on the right and the right part on the left. I feel I am close to a solution, but can't really point it out
from nokhwa.
Does not work on Macbook Air 2014 13". Same error.
from nokhwa.
Anyone have a work around for this? All cams in macOS are coming in as NV12 and hitting the same bad input buffer size issue
.
from nokhwa.
Damn, I just hit this problem, too.
Until NV12 support comes in v0.11, it's possible to use a crate like DCV Color Primitives to do the necessary conversion.
from nokhwa.
Hi, is there any workaround for this issue?
from nokhwa.
Hi, I'm seeing this issue as well. I think the problem is that AVfoundation returns frames in UYVY
format (https://wiki.videolan.org/YUV#UYVY), even if you tell nokhwa to request NV12
(or some other format).
Here is my current workaround to deal with this kind of data (note that this only works on Rust nightly). Alternativly, you could split the interleaved planes and use the aforementioned dcv-color-primitives crate for conversion.
#![feature(portable_simd)]
use std::simd::SimdFloat;
use std::simd::f32x4;
use rayon::prelude::*;
#[inline]
pub fn uyvy_to_rgb24(in_buf: &[u8], out_buf: &mut [u8]) {
debug_assert!(out_buf.len() as f32 == in_buf.len() as f32 * 1.5);
in_buf
.par_chunks_exact(4) // FIXME: use par_array_chunks() when stabalized (https://github.com/rayon-rs/rayon/pull/789)
.zip(out_buf.par_chunks_exact_mut(6))
.for_each(|(ch, out)| {
let y1 = ch[1];
let y2 = ch[3];
let cb = ch[0];
let cr = ch[2];
let (r, g, b) = ycbcr_to_rgb(y1, cb, cr);
out[0] = r;
out[1] = g;
out[2] = b;
let (r, g, b) = ycbcr_to_rgb(y2, cb, cr);
out[3] = r;
out[4] = g;
out[5] = b;
});
}
// COLOR CONVERSION: https://stackoverflow.com/questions/28079010/rgb-to-ycbcr-using-simd-vectors-lose-some-data
#[inline]
fn ycbcr_to_rgb(y: u8, cb: u8, cr: u8) -> (u8, u8, u8) {
let ycbcr = f32x4::from_array([y as f32, cb as f32 - 128.0f32, cr as f32 - 128.0f32, 0.0]);
// rec 709: https://mymusing.co/bt-709-yuv-to-rgb-conversion-color/
let r = (ycbcr * f32x4::from_array([1.0, 0.00000, 1.5748, 0.0])).reduce_sum();
let g = (ycbcr * f32x4::from_array([1.0, -0.187324, -0.468124, 0.0])).reduce_sum();
let b = (ycbcr * f32x4::from_array([1.0, 1.8556, 0.00000, 0.0])).reduce_sum();
(clamp(r), clamp(g), clamp(b))
}
#[inline]
fn clamp(val: f32) -> u8 {
if val < 0.0 {
0
} else if val > 255.0 {
255
} else {
val.round() as u8
}
}
(adapted from here: https://gist.github.com/arifd/ea820ec97265a023e67a88b66955855d)
As a sidenote, it looks like recent MacOS versions do not support transparent raw access to the camera outputs anymore, i.e. you will always be offered uyvy422
, yuyv422
, nv12
, 0rgb
, and bgr0
, regardless of what the camera actually supports. MJPEG streams will still work when requesting a resolution/fps mode only supported through MJPEG (as is common for higher resolutions on UVC cameras , but AVfoundation will handle the conversion to one of the listed pixel formats internally.
@l1npengtul, not sure what the current status is here (I'm a bit confused with the different branches) but I might be able to help you with debugging this.
Edit: Forgot to say that this is on a M2 MacBook Pro on Ventura 13.4.
from nokhwa.
Hi, I'm seeing this issue as well. I think the problem is that AVfoundation returns frames in UYVY format (https://wiki.videolan.org/YUV#UYVY), even if you tell nokhwa to request NV12 (or some other format).
see #151
from nokhwa.
Related Issues (20)
- Wrong `frame_rate` on Windows HOT 5
- Multiple AVFoundation cameras on macOS panics with an NSException
- yuv422 confusions? HOT 2
- Compile issues on v4l2_pix_format in linux kernel 6.2 HOT 3
- Can't compile on archlinux 6.5.5-arch1-1 HOT 1
- Unable to access capture card on windows HOT 1
- Use AVCaptureDeviceTypeExternal on macOS
- what FrameFormat::GRAY is?
- Incorrect pixel format HOT 1
- AVFoundation backend is not passing the frame format HOT 1
- Meta: Current Status, Progress and Future Plans for this Project HOT 12
- Support for macOS continuity camera HOT 3
- Possible unsoundness HOT 1
- Switching devices leads to assertion failure at the device list. HOT 1
- threaded camera deadlock on stop stream HOT 1
- [Windows] Panic when no cams are connected
- [Request] BGR and GBR support HOT 1
- [Web] Update `jscam` example to function correctly in the latest version
- Build fails on multiple versions on Pop_OS! HOT 2
- v4l camera indexing isn't necessarily accurate
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nokhwa.