The main use case of the transform API so far is end-to-end encryption. I could see other use cases like appending additional data to a frame. In both cases, we are talking of media being transformed by a sender, sent to a receiver that does the inverse transform to render the (potentially augmented) media.

It would be good to better understand the use cases you have in mind so that we break them down in requirements.
For instance, the use cases I mentioned above do not need 'the input or output to a transform go somewhere other than the normal path'. It would also help understanding what signals we are talking about and the best way to handle them.

Can you share more of the use cases?

The most obvious exampe is an outgoing stream that is encoded using a WASM or WebCodec encoder, and needs to be decoded using a WASM or WebCodec decoder.
This is an use case that is important for codec agiity - it permits both the introduction of new codecs outside of the global browser release cycle and the support of legacy codecs after browsers have dropped support for them.

In addition to feedback streams, this needs the capacity for telling the browser "don't bother with instantiating encoders/decoders, I'm doing this myself".

In addition to feedback streams, this needs the capacity for telling the browser "don't bother with instantiating encoders/decoders, I'm doing this myself".

Indeed, the pluggable codec use case seems like an additional API to me which would run prior WebRTC encoded transform in senders, and after in receivers.
For instance, WebRTC encoded transform API is solely based on RTCEncodedVideoFrame/RTCEncodedAudioFrame.
Pluggable codecs will probably also rely on AudioFrame/VideoFrame objects.

I could see some requirements derived for WebRTC encoded transform API from the pluggable codec use case.
For instance, it might be beneficial for encoded transform API to provide ways to read/write RTP headers.

Also, how would packetizer/depacketizer be handled for new or deprecated codecs? Codec-agnostic packetization?

Yes, we may have to design a completely new API rather than trying to merge this into the transform-supporting one. Would be great to have some commonality, though.
(There is an AI to merge the WebRTC encoded frame IDL with the webcodec encoded frame IDL. But the sequence of landings didn't let us depend on them at the time.)

@youennf "It would be good to better understand the use cases you have in mind so that we break them down in requirements."

[BA] The uses cases are described in WEBRTC-NV Use Cases. The VR/AR use case may involve substantial additions to the frame size, which is why adaptation to bandwidth or frame size is important.