Navigation Menu

Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Multi-pen support and persistent pointerId #353

Open
scottlow opened this issue Mar 2, 2021 · 15 comments
Open

Multi-pen support and persistent pointerId #353

scottlow opened this issue Mar 2, 2021 · 15 comments
Labels

Comments

@scottlow
Copy link

scottlow commented Mar 2, 2021

After digging around in issues for a while, I don't believe this has been raised before, but please feel free to dupe if so.

Lately, we at Microsoft have been thinking a bit about multi-pen scenarios for devices like Surface Hub. In cases like these, it is sometimes desirable to assign a pen a unique pointerId per top-level browsing context that remains static for the duration of the session regardless of whether the pen moves beyond the range of the digitizer. This would enable various scenarios:

  • "Remembering" the selected attributes of a specific physical pen within a web-based drawing application, allowing users to switch between physical pens of multiple colors/types/stroke widths/etc. just as they can do on a physical whiteboard
  • Allowing multiple people to collaborate at once using "different" pens. Today, I believe the best possible experience we can offer on the web is to let multiple people collaborate using a single pen style

Given various language in the PointerEvents spec such as

If a touch contact or pen stylus is lifted beyond the range of the digitizer, then it is no longer considered active

and

A user agent MAY recycle previously retired values for pointerId from previous active pointers, if necessary.

It may be that to maintain backwards-compatibility, such an idea would be best implemented as part of a new specification specifically designed for this use case, but I thought I'd float the idea here first to get folks' thoughts.

@patrickhlauke
Copy link
Member

Personally, I think it would make sense to have this as part of the PE spec, rather than a separate one. My very naive take would be that this is already possible/allowed by that "MAY" sentence you mention, and that it can be expanded to mention this particular scenario as a rationale for the "MAY".

I do wonder if there's privacy concerns though, and if this may need some opt-in on the part of the users (also, how long does it persist, what's a "session" in this context, etc).

But generally, I'd personally be in favour of this.

@patrickhlauke patrickhlauke changed the title Multi-pen support Multi-pen support and persistent pointerId Mar 2, 2021
@bmathwig
Copy link

bmathwig commented Aug 13, 2021

Hey @patrickhlauke

I think there is already some context around the lifetime of the pointerId in the NOTE section under pointerId. Our current line of thinking is that each pen would be mapped to an index, starting with 1, and each time that pen generated a PE the pointerId would be the same. If the document is closed or refreshed, the pointerId can be different for each pen. Behind the scenes, the implementation would map the Pen hardware ID to this index and keep track of it, limiting exposure of the unique Pen hardware ID from fingerprinting.

Do you think there is a backwards compatibility risk by doing it this way? Is there a case where someone would take a dependency on a "new-ID-per-event" system for creating unique SVG strokes? Is this use case exempt because of the verbiage:

Therefore authors cannot assume values convey any particular meaning other than an identifier for the pointer that is unique from all other active pointers

@bmathwig
Copy link

Alternative proposal to add new deviceId to PointerEvent:

partial interface PointerEvent {
    [SameObject] readonly attribute deviceId;
}

@patrickhlauke Can this thread be brought up at the next PointerEvent WG meeting to start a discussion about the best way to achieve this?

@patrickhlauke
Copy link
Member

@bmathwig tagged this as a future idea. currently we're in the process of trying to stabilise Pointer Events Level 3 for a possible publication this year.

@RByers
Copy link
Contributor

RByers commented Dec 6, 2022

Note that there's a proposal to add this to Chromium. It doesn't really matter to Chromium process if that's done by (as planned) an independent WICG spec, some future-branch PE L4 spec, or a separate PE extensions spec as we've done in the past for new additions while we're stabilizing a version of the spec. @patrickhlauke / @mustaqahmed any preference?

@patrickhlauke
Copy link
Member

@RByers hey stranger. I'd love to see this as part of L4 (or living standard, or however we want to set this up for future), so whichever approach would lend itself the best for that is ok with me. Maybe a WICG extension?

@RByers
Copy link
Contributor

RByers commented Dec 8, 2022

Hey man, good to have an excuse to post on PEWG again 😄. Sounds good, sounds like you are both thinking a tiny little WICG extension is simplest, so that's fine - we can move it into PE once L3 ships. FWIW I would love to see y'all move to a living standard model.

@mustaqahmed
Copy link
Member

Great to see the revived interest here! I agree with @RByers and @patrickhlauke re the WICG path until we are done with L3.

Let me bring back an old question from @bmathwig above, which I didn't get a chance to comment before:

Do you think there is a backwards compatibility risk by doing it this way? Is there a case where someone would take a dependency on a "new-ID-per-event" system for creating unique SVG strokes?

We don't know any use-case like this, and therefore re-purposing pointerId (as per the original proposal here) sounds like an equally acceptable solution to me. As @patrickhlauke mentioned, the "MAY" in the current spec would allow this already. Moreover, this could be interpreted as a generalization of the (existing) constant pointerId of mouse. Let's discuss this in the WICG repo.

@flackr
Copy link
Contributor

flackr commented Jan 23, 2024

therefore re-purposing pointerId (as per the original proposal here) sounds like an equally acceptable solution to me

I don't think re-purposing pointerId works because pointerId is expected to remain fixed for a given pointer but the hardware limitations suggest that for many devices the hardware id is not known until after it starts moving on the surface, and as such the device id would be -1 on the event stream until the point at which the hardware id is known.

I think we should discuss this proposal again at the PEWG. It has been implemented with a draft spec PR, and a proposal to ship in chromium.

@RByers
Copy link
Contributor

RByers commented Jan 23, 2024

Yeah that property of the hardware ID not necessarily being known at pointerdown time was, IIRC, the key aspect necessitating this design. If PEWG is now able to take this up, that's great! But we also shouldn't be afraid to incubate here - I don't think the stakes are very high (see my comment on the I2S).

@flackr
Copy link
Contributor

flackr commented Feb 1, 2024

This was discussed yesterday (notes) where the consensus was that we would like to see this identifier added to a common structure where other device specific customizations could go. The rationale is to avoid having a bunch of new optional members of pointer events at the base level.

@flackr
Copy link
Contributor

flackr commented Feb 1, 2024

Another question that came up is when (i.e. for what devices / OS's) this would apply. My understanding was that we should have a deviceId for any device in which the same pointer can be known from one contact to the next. I suspect this means we may not expect a deviceId for a touch coming from a particular device since a subsequent touch may or may not be from the same "pointer" (i.e. finger), in which case it may be that we want the name to suggest something about the contact, rather than "device" which suggests something about the device which was contacted with.

@sahirv
Copy link

sahirv commented Feb 15, 2024

Following up on the conversation from the deviceId I2S.

I think we should only have a deviceId when we can uniquely identify a particular pointer.

"pointer" (i.e. finger), in which case it may be that we want the name to suggest something about the contact, rather than "device" which suggests something about the device which was contacted with.

I think excluding touch from deviceId seems reasonable, at least to me, since I would not expect my individual fingers to be identified, but I can understand the rationale behind this question. I think my vocabulary isn't big enough or an alternative name :P ... it would need to emphasize that the pointer event has the same source device though.

@flackr
Copy link
Contributor

flackr commented Feb 27, 2024

One more question, is deviceProperties.deviceId a bit redundant? Should it just be something like deviceProperties.identifier, deviceProperties.id or maybe deviceProperties.uniqueId (hinting at the purpose) or would this be too confusing?

@sahirv
Copy link

sahirv commented Feb 27, 2024

I don't mind deviceProperties.id or deviceProperties.uniqueId. id works since we already know it's a property of the device. Likewise uniqueId is great as well, and is my main preference.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

7 participants