GithubHelp home page GithubHelp logo

Wrong `frame_rate` on Windows about nokhwa HOT 5 OPEN

Giotino avatar Giotino commented on June 12, 2024
Wrong `frame_rate` on Windows

from nokhwa.

Comments (5)

Giotino avatar Giotino commented on June 12, 2024

The positions of the numerator and denominator of MF_MT_FRAME_RATE (https://learn.microsoft.com/en-us/windows/win32/medfound/mf-mt-frame-rate-attribute) seem to be different between SetCurrentMediaType and GetCurrentMediaType.

With this ugly byte swap I get the correct frame_rate

let mut bytes: [u8; 8] = unsafe { media_type.GetUINT64(&MF_MT_FRAME_RATE) }
    .unwrap()
    .to_ne_bytes();
bytes.swap(0, 4);
bytes.swap(1, 5);
bytes.swap(2, 6);
bytes.swap(3, 7);
let frame_rate = u64::from_ne_bytes(bytes);
println!("Frame rate: {}", frame_rate as u32);

from nokhwa.

Giotino avatar Giotino commented on June 12, 2024

There is something strange going on, it seems that Windows is trying to correct the endianness of the two u32 (numerator and denominator).

With the "wrong" endianness (the one used by the library):

Bytes IN  SetCurrentMediaType: [0, 0, 0, 1, 0, 0, 0, 60]
Bytes OUT GetCurrentMediaType: [1, 0, 0, 0, 60, 0, 0, 0]

With the "correct" endianness:

Bytes IN  SetCurrentMediaType: [1, 0, 0, 0, 60, 0, 0, 0]
Bytes OUT GetCurrentMediaType: [1, 0, 0, 0, 60, 0, 0, 0]

from nokhwa.

Giotino avatar Giotino commented on June 12, 2024

I also fear that since the framerate is represented as numerator/denominator there could be devices that use framerates like 29.97 that can be represented as 2997/1000 and nokhwa is going to read 2997 as the framerate.

from nokhwa.

Giotino avatar Giotino commented on June 12, 2024

I just found out this issue was already reported #110 and #139

from nokhwa.

Giotino avatar Giotino commented on June 12, 2024

I think I get what's happening:
nokhwa is using the wrong endianness for the two u32 that represents the numerator and denominator, since

let fps = {
let frame_rate_u64 = 0_u64;
let mut bytes: [u8; 8] = frame_rate_u64.to_le_bytes();
bytes[7] = format.frame_rate() as u8;
bytes[3] = 0x01;
u64::from_le_bytes(bytes)
};

it's writing the two u32 in big-endian inside the u64:
[0, 0, 0, 1, 0, 0, 0, FRAMERATE] instead of [1, 0, 0, 0, FRAMERATE, 0, 0, 0].
Then Windows converts it to [1, 0, 0, 0, FRAMERATE, 0, 0, 0] (I don't know why, but it's trying to fix this mistake).
In the end Windows use the correct endianness when answering to GetCurrentMediaType and nokhwa read it wrongly (it reads the denominator instead of the numerator).

The fix required to make it works is to review the way that these number are built and parsed, also beacause currently it can't support more than 255 FPS. This also includes the part of code that build the list of available formats.

Also nokhwa should keep in mind that there might be some framerates that are not X/1 (like 2997/1000), I think here we have 2 solutions that doesn't require to rewrite the frame_rate handling:

  1. Return an error when the framerate is not X/1
  2. "Transform" A/B -> X/1 rounding X (es. 2997/1000 -> 30/1)

from nokhwa.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.