Comments (14)
We likely will need a surface to feature detect new encoder/decoder features as well. That doesn't seem like it would be covered by MediaCapabilities.
from webcodecs.
new encoder/decoder features
Steve, can you give an example?
from webcodecs.
@mounirlamouri fyi
from webcodecs.
I don't think we've discussed how specific we want encoder/decoder settings yet, but here's some examples I could think of (some of this may be suitable for MediaCapabilities):
- How much latency an encoder requires (measured in frames)
- What kind of SVC/simulcast a video encoder/decoder supports
- Available encoder keyframe modes (e.g., does it support forcing keyframes)
- Support for explicitly tagged golden frames (I've heard some hardware decoders don't support this?)
- Supported encoding modes: VBR vs CBR vs ??
- Post processing features
Disclaimer: Not a codec expert
from webcodecs.
At first glance I don't think we need a different API surface from MediaCapabilities. MediaCapabilities already has feedback on both decode and encode capabilities.
How much latency an encoder requires (measured in frames)
Is this a capability or something that can be measured by observing the behavior of the encoder?
What kind of SVC/simulcast a video encoder/decoder supports
Is this contentType: "video/H264-SVC" or similar values?
Available encoder keyframe modes (e.g., does it support forcing keyframes)
Support for explicitly tagged golden frames (I've heard some hardware decoders don't support this?)
Supported encoding modes: VBR vs CBR vs ??
MediaCapabilities may need a VideoEncodingConfiguration dictionary to be able to query for specific encoder capabilities.
Post processing features
How is this related to encoding? Can you give specific examples?
from webcodecs.
At first glance I don't think we need a different API surface from MediaCapabilities. MediaCapabilities already has feedback on both decode and encode capabilities.
Right. We want to use MediaCapabilities, but we may need things added to it.
How much latency an encoder requires (measured in frames)
Is this a capability or something that can be measured by observing the behavior of the encoder?
I think it's something that can be statically known and which you would like to know before creating the encoder, because there might be a range of delay options you can choose from and you might want to choose within that range.
What kind of SVC/simulcast a video encoder/decoder supports
Is this contentType: "video/H264-SVC" or similar values?
Available encoder keyframe modes (e.g., does it support forcing keyframes)
Support for explicitly tagged golden frames (I've heard some hardware decoders don't support this?)
Supported encoding modes: VBR vs CBR vs ??MediaCapabilities may need a VideoEncodingConfiguration dictionary to be able to query for specific encoder capabilities.
Yes I think that would make sense.
Post processing features
How is this related to encoding? Can you give specific examples?
No, it only applies to decode, not encode.
from webcodecs.
Would the next step then to be to craft a PR on what changes we'd like to see to Media Capabilities (as part of the explainer in this repository)? I can take a stab at that.
from webcodecs.
Yes, I think that would be a good next step. Thanks for helping.
from webcodecs.
Would the next step then to be to craft a PR on what changes we'd like to see to Media Capabilities (as part of the explainer in this repository)? I can take a stab at that.
Happy to review! You may also/instead want to file a bug with a rough outline of your proposed change. Guess it depends on how fast you are at spec-work (I'm slow) and how large the PR ends up being. Particularly for the big changes, discussion in an GH issue can be a bit easier and save you time re-writing spec-speak.
from webcodecs.
We've now added an IsConfigSupported() API for all encoders and decoders. Later integration with MC is not off the table, but not a priority at this time.
from webcodecs.
@steveanton WebCodecs doesn't explicitly support simulcast (e.g. multiple encoders need to be created). Question is whether explicit support is required or whether the implementation can optimize after noticing that there are multiple encodings utilizing the same track as input. SVC is supported using scalabilityMode
values and metadata.
from webcodecs.
Triage note: marking 'extension', as proposal here would be to add new dictionary members to the adjacent MediaCapabilities API. This doesn't change my stance mentioned earlier: I think WebCodecs has now solved most of this problem with it's various IsConfigSupported() methods.
from webcodecs.
@dalecurtis Can we close this issue?
from webcodecs.
I think it is covered by isConfigSupported(). Closing it
from webcodecs.
Related Issues (20)
- Define scope for w3c candidate recommendation HOT 3
- Candidate Recommendation tracking issue
- VideoFrame copyTo() behavior with non-RGBA/RGBX/BGRA/BGRX formats HOT 1
- VideoPixelFormat enum values do not follow casing rule guidelines HOT 2
- EncodedAudioChunkInit should probably also support AllowSharedBufferSource HOT 1
- Issue
- numberOfChannels/sampleRate check in AudioDecoderConfig/AudioEncoderConfig HOT 1
- Under what conditions should the ImageDecoder [[completed promise]] and ImageTrackList [[ready promise]] be rejected? HOT 3
- Spec differentiates between ImageDecoder initialization with unsupported non-image MIME type and an unsupported image MIME type HOT 2
- how to config videoEncoder that output frame is annexB not avcC? HOT 1
- `ImageDecoder` is `[SecureContext]` but related interfaces aren't HOT 2
- `ImageBufferSource` should be `[AllowShared]` HOT 1
- Announcement: Background Segmentation metadata entry to WebCodecs VideoFrame Metadata Registry HOT 2
- Why is the audio encoder output timestamp different from the input timestamp
- Video encoder bitrate control in case frame rate is not provided
- Video frames on demand HOT 1
- Error code: SIGSEGV HOT 3
- VideoFrame.timestamp and EncodedVideoChunk.timestamp HOT 6
- video encoder memory leaks HOT 1
- Include some frame info with `VideoDecoder` Errors? HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from webcodecs.