Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add "real-time" warning/note to MediaStreamTrackGenerator #20

Open
tguilbert-google opened this issue Apr 15, 2021 · 3 comments
Open

Add "real-time" warning/note to MediaStreamTrackGenerator #20

tguilbert-google opened this issue Apr 15, 2021 · 3 comments

Comments

@tguilbert-google
Copy link
Member

MediaStreamTrackGenerator does not specifiy any limits as to the speed at which data can be fed in. This can lead to problems if developers push 60s worth of audio data in 1s (see crbug.com/1184070).

There should be a note in the spec mentioning that developers are expected to feed tracks at a reasonable real-time rate.

@guidou
Copy link
Contributor

guidou commented Apr 23, 2021

We should indeed add this note to the spec.
Some mechanisms that could be used to notify applications about this type of problem:

  • Platform sinks can use existing error-reporting mechanisms (e.g., error event for media elements and maybe peer connectionss)
  • We can define a signal that can be exposed via MediaStreamTrackProcessor.readableControl

@guest271314
Copy link

This issue is still ripe. A note should be a sooner rather than later. An author of the WebCodecs specification author filed a Chromium bug due to the expectation that a MediaStreamTrackGenerator in a MediaStream set as srcObject of an <audio> element could be written to in a synchronous for loop without delay between calls and the audio would be output at the element at the same pace as delaying writes between calls to write() https://bugs.chromium.org/p/chromium/issues/detail?id=1260519.

@alvestrand
Copy link
Contributor

the input to the MSTG is a Stream. I would suggest that the note should say that the client of MSTG (or now: VideoTrackGenerator) needs to respect the high water mark of the stream, otherwise frames may be lost, and that one should expect the MSTG to process frames at a rate no slower than real time, but not much faster either.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants