We'd probably want a size restriction on these - maybe 32x32px is more than enough?

We'd probably want to make sure these images could be efficiently generated from a canvas. Since they're required to be small, perhaps canvas toBlob and using data URIs is good enough?

Discussed in today's group meeting and although pointerrawupdate and getPredictedEvents can reduce latency, based on Microsoft team testing the latency is still there and can hurt user experience. But it seems the API format proposed in this issue might not be the best fit for the use case. They will be looking into this to come up with an API that hopefully will address the latency issue better.

Yes, it is true that even with pointerrawupdate and prediction, there is still a gap in latency. I like the simplicity of specifying a cursor-like image for the UA to draw trailing the pointer, but in practice it will lead to undesirable artifacts, mainly because the stroke that is drawn by the app would not be taken into account.

Inking applications on the web already follow the pattern of ‘consume pointer event, produce strokes on a canvas’. This can be enhanced by things like pointerrawupdate, prediction, input delivered to workers, etc. without requiring web developers to change their underlying programming or data model. Similarly, I’d like to propose an additional enhancement to expose OS capabilities for drawing a 'cursor trail'.

Conceptually, we need to communicate the following information to the OS to enable this:
• Which pointer events were consumed and rendered as ink strokes
• Metadata around the ‘style’ of the drawn stroke, e.g. color, radius, tilt, pressure
Current thinking is that the metadata is limited in scope and is not designed to accommodate arbitrary shaders or complex images. Another option could be to expose supported metadata properties via a capabilities query.

We’re still working on the ergonomics and doing some prototyping, but I would love to get this groups feedback on whether a proposal along these lines make sense.

creating a reference for RFC TPAC: w3c/webappswg#10

We shared the following proposal at TPAC:

https://github.com/MicrosoftEdge/MSEdgeExplainers/blob/master/WebInkEnhancement/explainer.md

It's unclear whether this would fit under the PointerEvents WG charter, but we'd love to get this group of people's feedback on the proposal. I will post a link to WICG discourse thread once it is available.

Thanks!

@dlibby- I'm not quite sure about the charter stuff. From our discussions at TPAC I do believe we need to somehow tie this to the pointerevents and pointerid. But whether it fits under current PEWG charter or not I leave it up to @patrickhlauke to decide.

Regarding a feedback on the proposal, could you link here or in the explainer the native Windows API that supports the feature. I'd like to follow up with Android and ChromeOS team and see what they have for this.

The native Windows API does not yet exist and is currently work-in-progress — however, the proposed Web API exposure maps closely to how we expect this to be exposed for Windows (a method for setting 'style' properties, a method for communicating the last point rendered, and a method for communicating pen input points as they arrive).

The work to create a public API is about exposing this pen tip trail rendering functionality that is currently an implementation detail and encapsulated behind the InkPresenter and DirectInk family of APIs on Windows.

We should have more updates regarding the Windows API in the next couple of weeks.

We've updated the explainer based on some feedback from internal partners on the API surface, and would love to hear others' thoughts.

Updates include:

• The API now accepts a trusted PointerEvent, instead of x/y coordinates. This serves two purposes:

1. Remove a footgun for developers to send the wrong coordinates
2. Allow the User Agent to grab relevant properties off of the PointerEvent, in the case that more information will be needed in the future

• There is a property to set an element's bounds as a 'clip rect' for the delegated ink trail. This will scope the trail to the appropriate location of the page, and lets developers just set an element rather than dynamically calculating the 'clip rect' in the presence of scrolling, etc.
• There is now some indication of how much latency improvement an author might expect. This goes back to a suggestion by @smaug---- at TPAC that a browser could implement this fully within the application and provide some latency improvement on platforms that do not expose a native API (in fact this will be our plan of attack when prototyping in Chromium). If this is the case we thought it might be useful to authors to be able to take the expected improvement and choose whether or not to continue using the feature.

In terms of the OS API, it is still not yet available (turns out OS release timeframes are quite long). But the general shape is two new methods on a new interface - one method takes information about the application's last rendered ink point (very closely mirrors our setLastRenderedPoint() proposal), and another that allows the application to freely feed input to the compositor. Setting the last rendered ink point is associated with a specific swap/present of the backbuffer (the new visual and the swapchain are associated), so that if the application falls behind in ink rendering the window manager can continue to draw the input points that will continue to be fed.

I believe a preview SDK will be available sometime later this spring, at which point I'll link it here.

Given that there isn't even native API for this yet, this should probably be post-v3, or outside pointer events spec altogether.

Decided in today's PEWG call/meeting https://www.w3.org/2020/02/19-pointerevents-minutes.html that this is slightly outside of the charter/scope of PEv3. May move to a separate/adjacent spec. Link here from any new spec that comes out.