Disabling hardware acceleration #98
Comments
|
We don't have anything related to it and it's been up to the user agent for now. We could reuse the same member in the sender's parameters to achieve a similar result. I'm wondering if there's any signal happening when you use "prefer-hardware" and the user agent needs to fallback to a software implementation. |
|
hardwareAcceleration is an input parameter to Media Capabilities. |
|
In Chromium, WebCodecs "prefer-hardware" and "prefer-software" are interpreted as requirements (though the spec indicates that they should be thought of as a hint). "no-preference" is closer to "use hardware if you can, otherwise fallback to software". Looking at MediaCapabilties, VideoConfiguration does not currently include a hardwareAcceleration member. However, MediaCapabilitiesInfo (the output) does indicate whether the configuration is supported, smooth or power efficient. |
|
I think that Media Capabilities will not be able to reflect all the runtime failures during encoding/decoding for hardware. The fallback mechanism will take care encoding/decoding job if hardware fails during runtime. |
|
I went through my list-of-interesting bugs and found quite a few which were related to hardware encoding or decoding. Hardware H264 Encoder is being used instead of OpenH264 on MacOS resulting in poor frame rate / pixelated video Consider adding support for hardware vp8 decoding on Windows. Disabling hardware acceleration silently fails to encode camera frames in WebRTC on macOS If the stream obtained via HTMLVideoElement.captureStream() is sent on peer connection, it does not send any actual data Green Screen on WebRTC (w/ H264 Codec) Recorded Videos when Hardware Acceleration is disabled VP9 decoder shows a green strip at the bottom (Windows only) Encode fps drops sharply after pc.replaceTrack, due to FrameDropper kicking in from too many hugesFrames encoded, when encoding H.264 on MacOS WebRTC video stream shows green and garbled in chrome 91 Beta Resolution cannot ramp-up to 720P on Windows Chrome 91 with HW acceleration enabled VP8 encoder issue when hardware acceleration enabled on mobile device Windows H264 encoder may freeze Android MediaCodec Encoder requires 16-pixel alignment VP8 encoder issue when hardware acceleration enabled on mobile device iPadOS 15 / iOS 15 unable to decode VP9 stream WebRTC Output FPS Drops to 0 and Does Not Recover Android HW Encoder produces corrupted images (also in Google Duo) Chrome Canary - WebRTC RTCPeerConnection.close() not returning WebRTC framesSent plummets for no obvious reason Corrupt WebRTC video from VP9 hardware decoder on Intel/Windows Safari: VP9-SVS no video stream from remote peer on some devices Apple Silicon: Black screen share on remote side of peerconnection H264 HW encoding/decoding cause breaks the picture H264 HW encoder produces 1 key frame per second on webrtc Chrome version 98(which introduces HW encoding) degrades sent resolution and increases keyframe generation https://bugs.chromium.org/p/chromium/issues/detail?id=1295815 |
|
@fippo That is a very impressive list of bugs! @Orphis @alvestrand Are we looking to modify the negotiation process (e.g. filter codecs emitted by Parameter setting could work in a scenario where both hardware and software support is available and the goal is to prefer one over the other. But if the preferred codec is only supported in hardware, An extension to |
|
Sadly I missed the meeting. Here is what I would do (I actually have a chrome CL for it even), extending RTCRtpSender and Receiver with When called before creating any peerconnection this disables hardware encoding and decoding respectively. This is a sledgehammer and a terrible API but making it
is intentional. This should be covered by appropriate usage metrics, ideally ones allowing to identify large users and then asking them why they feel a need to use it. This API should not be an excuse not to fix bugs ;-) |
|
Meeting minutes and video are available here. As noted in the minutes, the high level question was whether this is something the browser should handle "under the covers" (e.g. via a whitelist) or something that the application should have control over. If the application has the ability to disable hardware acceleration, how does it know when to do so? Disabling all hw acceleration, or hw acceleration for a particular codec, may be a blunt instrument. If there is a bug in a particular codec and set of hardware, how does the application know whether it is running on that hardware? It would need to access information about the device which would also be useful for fingerprinting. That concern is not present with a whitelist. |
|
I don't believe a whitelist makes sense across multiple browser vendors, developers are already having to write "if safari and 15.1... disable h264" kinds of conditions.... I like @fippo 's idea of "an emergency scenario API" that doesn't give fine grained control but does give the control to the end developer allowing them to do "if chrome 100 disable hardware acceleration" - at the end of the day developers are currently stopped from being able to fix a failing web app and sometimes spend weeks waiting for a browser release - why would a whitelist get changed any quicker - we're still reliant on someone from a browser agreeing the issue exists and removing something from a whitelist. Let the developer take control but do it in a way that doesn't allow fingerprinting. |
In case this information is useful, at Threema we already have a exclusion list of devices with broken hardware codecs. (Not surprisingly, all of them are made by Samsung.) https://github.com/threema-ch/threema-android/blob/557b69f33dd1db96f58e41f6602e32522470f53e/app/src/main/java/ch/threema/app/voip/Config.java#L59-L64 Additionally we also have an exclusion list for hardware AEC: https://github.com/threema-ch/threema-android/blob/557b69f33dd1db96f58e41f6602e32522470f53e/app/src/main/java/ch/threema/app/voip/Config.java#L35-L41 Maybe an API for disabling hardware acceleration for video codecs could be extended to also support disabling hardware AEC? |
Chrome has had a variety of issues with hardware encoders: w3c/webrtc-extensions#98 (comment) We'll disable hardware encoders for now so that we can get consistent behaviour everywhere. Once we've updated to a modern libwebrtc we can reenable them. This also makes it so that we can get resolution scaling on H264 without having to cherrypick patches from upstream libwebrtc. Differential Revision: https://phabricator.services.mozilla.com/D144773
Chrome has had a variety of issues with hardware encoders: w3c/webrtc-extensions#98 (comment) We'll disable hardware encoders for now so that we can get consistent behaviour everywhere. Once we've updated to a modern libwebrtc we can reenable them. This also makes it so that we can get resolution scaling on H264 without having to cherrypick patches from upstream libwebrtc. Differential Revision: https://phabricator.services.mozilla.com/D144773
|
https://www.w3.org/2022/09/12-webrtc-minutes.html#t18 -- TPAC minutes (so I can find them). PR should be incoming soon. |
|
great! ！Looking forward to this feature coming soon， We have encountered several video blurring problems caused by hardware decoding, and finally can only be solved by manually turning off hardware acceleration through chrome://flags， |
Please file a chrome bug. In particular for Intel they are very interested in fixing issues that pop up. |
|
the issue fixed via #128 ? |
|
@bdrtc, have you filed a related Chromium bug regarding the hw encoding problems you found? |
|
|
Addressed by PR #128 |
WebCodecs has a
hardwareAccelerationmember in theVideoConfigdictionary.Is there a way to express a similar preference in WebRTC?
The text was updated successfully, but these errors were encountered: