Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add API to request key frames #125

Merged
merged 5 commits into from Jan 27, 2022
Merged

Add API to request key frames #125

merged 5 commits into from Jan 27, 2022

Conversation

youennf
Copy link
Collaborator

@youennf youennf commented Nov 19, 2021

This includes 3 different APIs:

  • An API to generate a key frame from a RTCRtpScriptTransformer on sender side
  • A corresponding API in RTCRtpSender. This API shares the same algorithm as the first one
  • An API to request a key frame (through FIR) from a RTCRtpScriptTransformer on receiver side

Preview | Diff


Preview | Diff

@youennf
Copy link
Collaborator Author

youennf commented Nov 19, 2021

This PR is based on w3c/webrtc-extensions#37, but has some differences.
For instance, here, the promise returns undefined. The rejected errors are not exactly different either.
@aboba, could you take a look?

@youennf
Copy link
Collaborator Author

youennf commented Nov 19, 2021

Bot error should be fixed by #123

@youennf
Copy link
Collaborator Author

youennf commented Nov 19, 2021

Fixes #124

index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
index.bs Outdated Show resolved Hide resolved
…t the first enqueued key frame since generateKeyFrame was called

Also make the possible error checks synchronously instead of within the in parallel steps.
webkit-commit-queue pushed a commit to WebKit/WebKit that referenced this pull request Dec 20, 2021
…oposal

https://bugs.webkit.org/show_bug.cgi?id=234429

Reviewed by Eric Carlson.

Source/WebCore:

Update implementation according w3c/webrtc-encoded-transform#125.
This means adding a specific method to send a FIR, a specific method to generate a key frame on sender side.
This also means improving the error handling and the promise resolution timing.
RID support is not yet available until we can properly pipe that information down to encoders.

Test: http/wpt/webrtc/audiovideo-script-transform.html

* Modules/mediastream/RTCRtpScriptTransformer.cpp:
* Modules/mediastream/RTCRtpScriptTransformer.h:
* Modules/mediastream/RTCRtpScriptTransformer.idl:

LayoutTests:

* http/wpt/webrtc/audio-video-transform.js: Added.
* http/wpt/webrtc/audiovideo-script-transform-expected.txt: Added.
* http/wpt/webrtc/audiovideo-script-transform.html: Added.
* http/wpt/webrtc/context-transform.js:



Canonical link: https://commits.webkit.org/245414@main
git-svn-id: https://svn.webkit.org/repository/webkit/trunk@287258 268f45cc-cd09-0410-ab3c-d52691b4dbfc
annulen pushed a commit to qtwebkit/qtwebkit that referenced this pull request Dec 20, 2021
…oposal

https://bugs.webkit.org/show_bug.cgi?id=234429

Reviewed by Eric Carlson.

Source/WebCore:

Update implementation according w3c/webrtc-encoded-transform#125.
This means adding a specific method to send a FIR, a specific method to generate a key frame on sender side.
This also means improving the error handling and the promise resolution timing.
RID support is not yet available until we can properly pipe that information down to encoders.

Test: http/wpt/webrtc/audiovideo-script-transform.html

* Modules/mediastream/RTCRtpScriptTransformer.cpp:
* Modules/mediastream/RTCRtpScriptTransformer.h:
* Modules/mediastream/RTCRtpScriptTransformer.idl:

LayoutTests:

* http/wpt/webrtc/audio-video-transform.js: Added.
* http/wpt/webrtc/audiovideo-script-transform-expected.txt: Added.
* http/wpt/webrtc/audiovideo-script-transform.html: Added.
* http/wpt/webrtc/context-transform.js:


git-svn-id: http://svn.webkit.org/repository/webkit/trunk@287258 268f45cc-cd09-0410-ab3c-d52691b4dbfc
@aboba
Copy link
Contributor

aboba commented Jan 4, 2022

@youennf @alvestrand Do we understand how we can ensure key synchronization when a new participant joins the conference? In this situation, the sender may desire to generate a key frame and then apply a new encryption key (and keyId) to that keyframe.

The updated key needs to be applied to the new keyframe, not to a previous or subsequent frame (which would lead to having the new participant being unable to decrypt and decode video until another keyframe update). If a keyframe is encrypted using the old encryption key, then a newly joined receiver will not be able to decrypt it.

There are a number of ways in which the sender can generate a key frame and then ensure that a key update is applied to that newly generated key frame. The current PR does not make it clear how this would be achieved.

One way this might work is to have generateKeyFrame() return a timestamp which is then provided to setEncryptionKey(). It may also be possible to avoid a race condition without passing a timestamp.

@aboba aboba merged commit f683566 into w3c:main Jan 27, 2022
github-actions bot added a commit that referenced this pull request Jan 27, 2022
SHA: f683566
Reason: push, by @aboba

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants