This was previously discussed in whatwg/html#4876 and given the last comment there I'm somewhat surprised it's now resurfacing here.

@annevk Looking at the explainer, this new behavior is opt-in

As this actually seems to touch two different areas - there are even two different explainers - we should probably split this up into two issues

I split up part of this into #507

There are a lot of good use-cases here: whatwg/html#4876 (comment)

I think these would be good to add clearly to the explainer

@snianu Given the feedback from @annevk above, is this proposal a conflict resolution request, or is there a misunderstanding and this review does not touch on the issue mentioned? Would be good to have a clarification.

Thank you everyone for all the feedback!
@cynthia @annevk This issue specifically talks about inputMode attribute and this comment was about deprecating inputMode="none"'s special behavior of hiding the VK which we believe is conflating two different states and doesn't cover some of the scenarios mentioned in the explainer. This feature gives explicit control of the VK to the web authors. It also defines an attribute that differentiates between the custom (virtualKeyboardPolicy="manual") and default(virtualKeyboardPolicy="auto") VK behavior. When virtualKeyboardPolicy is "manual", inputMode ="none" 's hiding of the keyboard state is ignored, but the default VK layout is kept intact. Basically, with virtualKeyboardPolicy attribute, the web authors could control the VK behavior without affecting the layout of the VK which would be controlled by the inputMode attribute. These two states (visibility and layout of the VK) are kept separate in this proposal so web authors can use it to facilitate more complicated scenarios on their sites.
The new API can be feature-detected by checking for the existence of the virtualKeyboard property on the navigator object.

@kenchris This section talks about the range-peeking scenario in Excel that sort of merges the two use cases that were mentioned in this comment, but I agree that we should explicitly mention that in the explainer. I'll create an issue in our repo and address that in the explainer.

@snianu from what Ryosuke stated it seems unlikely to change however, which is more relevant I'd think.

(@annevk's referring to this comment of @rniwa's from February.)

@rniwa could you please clarify your objection from this comment?

I believe you are objecting to any deprecation or change in behavior for inputmode=none.
This proposal doesn't suggest we deprecate inputmode=none.

I think the root of the confusion is that inputmode=none has some overlap with the VK APIs (but isn't a replacement) so at one point it was suggested that we could deprecate inputmode=none since we would have the more capable VK API. @rniwa is saying that wouldn't be acceptable as inputmode=none has been promoted and is being used to dismiss the keyboard in some scenarios.

We therefore abandoned the idea of deprecating inputmode=none (in response to @rniwa's comment), but that doesn't mean we don't need the VK API. The VirtualKeyboard interface and the virtualKeyboardPolicy attribute provides authors with the ability to:

1. Request that the keyboard be hidden
2. Request that the keyboard be shown
3. Handle an event for when the keyboard changes its visibility and adjust to the size and position of the keyboard
4. Prevent the visual viewport from being resized due to the appearance of the VK

inputmode=none only addresses the first of those capabilities and comes with side effects that are discussed in the same github issue (see here).

Hope that helps clarify things. I don't believe there is any conflict here.

I've been on a medical leave. Hopefully @whsieh can clarify our position here.

Given the comments from @BoCupp-Microsoft above, we'd like to hear if there is still strong objection with regards to the new proposal, as we don't find it as controversial as we initially thought. Can @whsieh or someone else comment on the objection points?

Sorry for the delay; I have a couple of questions below:

For (3) — it wasn’t totally clear to me why the visual viewport API isn’t already sufficient for this purpose. Is it because the VisualViewport API isn’t compatible with devices with multiple screens, when the software keyboard only takes up space in one of the screens?

For (4) — I would like a bit more clarity on what it means to prevent the visual viewport from being resized due to the appearance of the virtual keyboard. Does that mean the visual viewport should be “pushed upwards” (and retain its size) when the virtual keyboard is brought up? Or that the visual viewport should overlap with the bounds of the software keyboard?

For (4) — I would like a bit more clarity on what it means to prevent the visual viewport from being resized due to the appearance of the virtual keyboard. Does that mean the visual viewport should be “pushed upwards” (and retain its size) when the virtual keyboard is brought up? Or that the visual viewport should overlap with the bounds of the software keyboard?

The latter. The visual viewport will overlap with the bounds of the virtual keyboard when the author sets navigator.virtualKeyboard.overlaysContent = true.

For (3) — it wasn’t totally clear to me why the visual viewport API isn’t already sufficient for this purpose. Is it because the VisualViewport API isn’t compatible with devices with multiple screens, when the software keyboard only takes up space in one of the screens?

Philosophically the visual viewport API isn't sufficient because when then the keyboard overlays the content the visual viewport remains unchanged. IMO the visual viewport API should not describe events that don't affect the size of the visual viewport.

Additionally, as you mention, the visual viewport describes a rectangular viewport, which is problematic for virtual keyboards on dual screen devices because those would create an L-shaped visual viewport if we don't want to waste space as shown in this explainer.

Do we want to have a virtual F2F discussion on this topic? I'm willing to meet up just with the people on this thread or at a larger venue during virtual TPAC. Let me know if anyone has suggestions for a venue.

Also, I think we're trending towards agreement. I'm interested in finding collaborators that would like to co-author a spec. Please let me know if your interested. Past experience writing specs would be a big plus. :-)

It seems worth discussing this as a breakout session: https://www.w3.org/wiki/TPAC/2020/SessionIdeas

Sounds like this discussion is going to happen at TPAC. That's great. Please let us know if you want someone from the TAG to join. I'll bump this issue until after TPAC and we can pick it back up then.

@rniwa @torgo @kenchris @snianu @cynthia @whsieh @bokand I've proposed a breakout session here called Virtual Keyboard Control. Right now I have the session marked so that it will be scheduled during the 2pm UTC golden hour that was suggested on either Wednesday, Thursday or Friday of the breakout week. Hopefully that will work for anyone that would like to attend. If not, let me know and I can always just schedule something outside of TPAC.

@BoCupp-Microsoft Can you post a summary of the relevant discussion at TPAC?

Meeting minutes: https://www.w3.org/2020/10/30-editing-minutes.html
Summary of the discussion:
We discussed about the VirtualKeyboard APIs and how it affects UX on both dual and single screen devices. The presentation was mainly focused on describing the problems faced by web authors while dealing with virtual keyboard(VK) on touch devices where the VK is the primary source of text input for users. We showed a demo of the VK APIs and how the authors could use it to provide a much better UX experience on their sites.
Demo page: https://bocupp-microsoft.github.io/VirtualKeyboard/geometrychange.html
Slides: https://cuppfamily-my.sharepoint.com/:p:/g/personal/bo_seattlecupps_com/EVoI7Hp5An1HokezOTE3hK0BLMT2UsL1k7-ryO8-8efv6w?e=EplB9S

Issue discussions:

• Concerns about VK APIs being abused and sites behaving poorly because of that:
  We currently have sticky user activation and active document as a requirement to restrict showing the VK. VK APIs are also an opt-in mechanism where authors have to explicitly indicate that they would like to control the VK behavior. Currently the show/hide behavior can be abused by using inputMode property.
• Is navigator the right place?
  We didn't get a chance to discuss this in detail, but I haven't seen any objections in this issue after the latest response.
• Relationship with the VisualViewport API and concerns in general regarding creating a separate API to handle overlaying of content in response to VK show/hide:
  We had extensive discussions on this issue. We generally agree that having a separate event describing the overlay property and geometry of the VK helps the authors make better decision as to how to position specific parts of the webpage in response to VK occluding the content of the webpage.
• Have VK show/hide APIs return a promise
  We all agreed that returning a promise when authors call show/hide helps to test the APIs and also make decisions when the calls failed to show/hide the VK. This needs more discussion as we are not entirely sure that returning an error code helps the authors in a meaningful way.
• We are also looking for collaborators to write a spec for these APIs.
  @BoCupp-Microsoft Please let me know if I missed anything.

I also saw that you added support for using CSS environment variables for the geometry in addition to the JS API 👍

Based on the discussions above - we took this up in the plenary call this week and decided we're happy to see this move forward. Thanks for bringing this to our attention!

Hello TAG,

Based on the comment from @cynthia here, could you update the labels on this issue to include Resolution: satisfied?

Thanks for your help on this!
Bo

Tagging a couple other members so this comment gets seen: @atanassov @kenchris

Done!