Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add a high-quality face/body tracking API to discourage poor/discriminatory implementations #1

Closed
steely-glint opened this issue Nov 9, 2020 · 4 comments

Comments

@steely-glint
Copy link

Two of the usecases for this (excellent) API depend on face and/or body tracking.
(Funny hats and Background removal)
It has been seen that many of these algorithms have been trained on inadequate datasets and perform badly for certain groups of users.
This API must offer high quality implementations of face and body tracking so that this problem can be avoided before it happens.
The site developer can use/load their own implementations of course, but the presence of a high quality implemenation will raise the bar and discourage substitution with worse methods.

@alvestrand
Copy link
Contributor

I'm not sure about this. It would seem like a separable concern, and I tend to be very supportive of concern separation.
If one were to specify such an API that is defined in terms of a stream of VideoFrame, that would offer instant pluggability without any spec linkage needed (and a native implementation could save some memory copies by operating on the internal representation and only surface the actual tracking result).

But this may be out of webrtc scope - is anything like this being pursued in (for instance) the AR/XR groups?

@steely-glint
Copy link
Author

I don't beleive it is separable. This proposed API claims to enable Funny Hats. If delivered without a non-discriminatory face tracking implementation it will cause lazy/discriminatory sites to proliferate.

Have a listen to the first 5 mins of a recent podcast I co-host https://distributedfutu.re/#episode56 for a sense of the scale of the problem that already exists: 'not working for darker skin tones .... Has become the norm now for me' says my co-host.

@alvestrand
Copy link
Contributor

@steely-glint I have added this to the agenda for Wednesday Dec 2's WG meeting. I'd like to get a firm grasp of "next steps".

@guidou
Copy link
Contributor

guidou commented Jun 18, 2021

Closing, since my understanding is that we have decided to work on this and other potential platform transformations in a separate document (maybe mediacapture-extensions initially).

@guidou guidou closed this as completed Jun 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants