Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Richer negotiation re: address redaction? #842

Open
samuelweiler opened this issue Feb 28, 2019 · 45 comments · Fixed by #955
Open

Richer negotiation re: address redaction? #842

samuelweiler opened this issue Feb 28, 2019 · 45 comments · Fixed by #955
Labels
Formal Objection Future Candidate Feature i18n-tracker Group bringing to attention of Internationalization, or tracked by i18n but not needing response. privacy-needs-resolution Issue the Privacy Group has raised and looks for a response on.
Milestone

Comments

@samuelweiler
Copy link
Member

samuelweiler commented Feb 28, 2019

As I think about how "billing addresses" are used in practice in the US: sometimes portions of them (e.g. only a postal (ZIP) code) are used by card issuers for validation. Sometimes merchants want complete addresses for other reasons. Does it make sense to have a requester explain exactly what address elements it wants, and why?

Consolidated explanation of my concern below at:
#842 (comment)

@ianbjacobs
Copy link
Collaborator

Hi @samuelweiler,

Can you say more about the use cases?

Our initial choice of redaction was driven by the use case of "tax computation." At least in version 1, that seems like a good starting point. After consideration of more use cases, we can see whether some merchant/user negotiation would add valuable.

Ian

@samuelweiler
Copy link
Member Author

In the US, some merchants, particularly ones that are not shipping a physical product, request (only) a "credit card billing ZIP code", which appears to be used as part of the card acceptance validation. I'm imagining that it would be convenient - and lead to less transaction abandonment - if the merchant could ask for only the info needed (e.g. just the post code) and not anything else.

I also imagine that different regions have quite different patterns of practices like this, hence the suggestion for a more generalized negotiation mechanism.

@npdoty
Copy link

npdoty commented Feb 28, 2019

I'm not sure user-driven negotiation is particularly useful here (although generally I like that as a strategy) since the user doesn't typically know or set the rules for required data to complete a payment or shipment. But merchant-driven minimization of information would definitely fall under the principle of data minimization, and that can do things like improve user comfort and even encourage payments. It sounded like this feature had been proposed for a future version and maybe Ian could link us to the corresponding issue.

@samuelweiler
Copy link
Member Author

npdoty: the problem I see is that merchants often desire more data than is, in fact, required. Making easier for merchants to get the data (by defaulting to sending it) is a lose for the human sending the payment.

@npdoty
Copy link

npdoty commented Feb 28, 2019

@samuelweiler but are end users in a good position to know what information is required, or to do anything besides refuse payment altogether if the site wants information that might not be necessary?

I think there are some times that negotiation might be a good approach even though it can be complicated to specify and implement, but I think those are cases where users have a lot of insight and there can be progressive levels of functionality based on level of information. But payment seems complex, externally regulated and mostly binary, though perhaps the analog example of haggling over prices would be a non-binary example.

@samuelweiler
Copy link
Member Author

@npdoty Contemplate the analog example of going into a Radio Shack (back in the day), Staples (now), or many grocery stores. They'll ask for a phone number, but they're still willing to make a sale without one. I can imagine an online merchant preferring to have some bit of data but being willing to make do without.

@ianbjacobs
Copy link
Collaborator

Hi @samuelweiler and @npdoty,

I mentioned on the call today that the WG has considered the idea of an array provided by the merchant that says "I don't want response data fields x and y for payment method P."
#97

For the moment the industry demand to exclude CVC has been small, so there was little appetite to include this as a standard feature in Payment Request available to all payment methods.

One could also define this feature at the level of a particular payment method. As an example, for Basic Card [1] one could enable the merchant in the request data to say "Please don't collect the CVV." However, the demand for this has been sufficiently small that we have not yet chosen to do so.

I generally think of it this way:

  • Experiment at in payment method definitions.
  • When there is a clear demand for the same functionality across payment methods, elevate it to Payment Request API.

So a future version of Basic Card (or any other payment method) could try out this feature without requiring changes to Payment Request API.

Ian

[1] https://www.w3.org/TR/payment-method-basic-card/

@samuelweiler
Copy link
Member Author

Acknowledging that I'm not swapped in on the payments markets elsewhere in the world, I imagine there's enough variation that this 'wait for clear demand' approach will lead to some (unhelpful, unwanted, and possibly harmful) convergence, driven by the US payment market. Or, perhaps even less desirably, to less adoption of this technology worldwide. Building in the flexibility from the start seems like a better approach.

@adrianhopebailie
Copy link
Collaborator

We should distinguish between data required for the payment and the data requested by the merchant as part of checkout.

E.g. Billing address is actually needed to authorize a card payment in many markets but we also provide a mechanism for the merchant to ask for an email address, phone number etc.

In this API we don't have any influence over payment specific data (either in the request or response). Billing address is used in the card payment flow but is:

  • unique to certain territories
  • used as a fraud indicator along with potentially many other data points that are decided by the card industry

If there is a desire to make it possible that data like email and phone number can be requested but marked as optional then we should look at changing the PaymentOptions dictionary so that the fields are not boolean but rather some enum like 'required' | 'optional' | undefined.

@ianbjacobs
Copy link
Collaborator

@adrianhopebailie,

I like your idea (for PR API after v1) and also would extend that functionality to payment method response data (relates to issue #97).

Ian

@ianbjacobs ianbjacobs added this to To do in Payment Request 1.1 via automation Mar 4, 2019
@ianbjacobs
Copy link
Collaborator

@samuelweiler, for now I am labeling this a "feature after v1" and we'll discuss it at our April FTF meeting. Thanks!

@samuelweiler
Copy link
Member Author

I think this is pretty clearly related to the features being added in this revision: https://lists.w3.org/Archives/Public/public-payments-wg/2019Feb/0003

I would like to see this issue addressed at the present revision.

@ianbjacobs
Copy link
Collaborator

Hi @samuelweiler,

I will consider this a Formal Objection raised by you (arising through PING review) and carry it forward to the Directory.

Ian

@marcoscaceres
Copy link
Member

marcoscaceres commented Mar 5, 2019

@samuelweiler, @npdoty, what I've done thus far is heavily redact the billing address information sufficiently so it works with fraud detection systems, and allow for tax calculations.

Tax calculations, performed when the payment instrument changes:
w3c/payment-method-basic-card#67

Fraud prevention:
#846

I share the concerns about requestBillingAddress, but instead of an enum, as @adrianhopebailie suggests, how about we use either a boolean or an array?

So:

  • requestBillingAddress is false (this is default): Merchant gets nothing.
  • requestBillingAddress is true: Merchant gets redacted billingAddress sufficient for fraud prevention and tax calculation.
  • requestBillingAddress is Array of members that the merchant needs (and redacted list still applies). So, "requestBillingAddress": ["recipient", "postcode"] would ignore "recipient" because it's on the redact list, and thus only gives the merchant back the post code. Gives merchant more control if they need to do tax calculations and/or fraud prevention checks.

As this is a significant change to the API, I don't think we should do this in V1. We should do this in v1.1 tho. Using the array should be backwards compatible with the 1.0, in as far as an array will be treated as true by the 1.0 API.

@marcoscaceres
Copy link
Member

marcoscaceres commented Mar 5, 2019

The other practical thing we could do for privacy is make the redactList mandatory. Right now, it's optional... and some UAs literally leak all the things: #648 (comment)

@samuelweiler
Copy link
Member Author

I've presented a use case above where only part of the address is needed (modeled on what I see some sites doing now), and I've postulated that these needs will be different in different markets around the world. I'm opposed to making a change (at v1 or otherwise) that defaults to increasing the amount of information over what is sent now, with no obvious way for payer and payee to negotiate about this information exchange.

@marcoscaceres If you don't want to make changes at v1, perhaps then you omit requestBillingAddress from v1?

@ianbjacobs
Copy link
Collaborator

@samuelweiler, we added this functionality because there is otherwise a risk that the displayed total is incorrect, which makes the API unusable.

@marcoscaceres
Copy link
Member

What @ianbjacobs said. It’s not practical to remove it because of tax calculations. Note that it’s also not very different from having a regular web form that is automatically filled by the browser.

I do understand the data sharing concern: but by the user taking explicit action to “pay”, there is a reasonable expectation of negotiation and understanding in any transaction that billing address may affect tax, that shipping address will affect shipping cost (along with shipping options), and where credit cards are involved, that merchants have a right to protect themselves against fraud, etc.

@samuelweiler
Copy link
Member Author

@marcoscaceres We disagree re: reasonable expectations, transparency, and negotiation. e.g. in the US, sometimes city is sufficient to determine tax. Perhaps even just state is enough - there is no need for specific street info. (And, also in the US, I tend to see sales tax determined by "shipping address", when something is being shipped.) There is lots of grey area here, and the API described is not modeling the reality well. And it's shipping more information than is needed in all cases.

@ianbjacobs
Copy link
Collaborator

Hi @samuelweiler, can you say more what you mean by this: "And it's shipping more information than is needed in all cases."

@marcoscaceres
Copy link
Member

there is no need for specific street info.

Precisely why street info is removed by the redactList. Perhaps something that was overlooked during your review?

There is lots of grey area here, and the API described is not modeling the reality well.

Respectfully, I find this quite baffling a statement to make. We are literally working with a global set of merchants, credit companies, and other financial tech representatives in this working group for years to come up with the design.

Additionally, the API and precursors (e.g., Apple Pay) have been shipping in browser for over 2-3+ years - proving themselves adequate around the world for the purpose of purchasing both digital and physical goods. None of the membership or implemeters have come back and raised the address structures and contents as “not modelling reality well”?

I also feel the discussion is getting off track a little. We need to assume the shipping and billing addresses do model reality: but what is of importance to the discussion is limiting what information of the address is exposed. From our research and imput from the WG members that what we are redacting strikes the right balance - and we allow user agents to go further, such as user agent further redacting post codes to further protect user privacy.

@marcoscaceres
Copy link
Member

marcoscaceres commented Mar 8, 2019

@samuelweiler, my apologies, in my mind I thought we'd merged the following already (which redacts the addressLine):
https://github.com/w3c/payment-method-basic-card/pull/67/files

I've also made the redactList mandatory:
#846

Does the above alleviate your concerns?

@samuelweiler
Copy link
Member Author

samuelweiler commented Mar 18, 2019

Restating my concern, which will hopefully answer @marcoscaceres and @ianbjacobs's questions.

I am concerned about "oversharing" - when an API, by default, automatically sends more identifying information than is sent now. This API is sending straight up PII (personally identifiable information). Because of that, it should meet a high bar.

For the two use cases presented - calculating sales tax and shipping costs before completion of a transaction - this API overshares in some cases.

Below are some examples that speak to what information is minimally necessary for the use cases. I could also point at specific merchants that ask for less info (since my concern is about what this API sends v. what is sent now), but looking at necessity of the information is probably more helpful in understanding the problem.

  • In the US, billing address is typically not needed for sales tax calculations - those are based entirely on shipping address. Sending any portion of a billing address - at least pre-completion - is oversharing.

  • In New Hampshire, Delaware, Oregon, Montana, and Alaska - US states which have no sales tax - the only portion of a shipping address needed for sales tax calculation is the state. Sending city or post code pre-completion is oversharing.

  • Also in the US, shipping costs are often flat. For the shipping use case, then, sending a city, post code, and even state pre-completion might be oversharing.

  • Some US merchants offer flat shipping rates to the "lower 48". All these need to calculate shipping cost is "US" and "not Alaska or Hawaii". Post code and city are oversharing, and arguably even the specific state is oversharing.

These examples are all from the US - a thriving market for online commerce. And yet even with the mitigations in place so far, this API still overshares for these US regions. I imagine that similar oversharing happens when we look at other jurisdictions. What portion of which address is needed to calculate shipping cost or sales tax in the Cayman Islands? Do those calculations need anything more than the country? What about in Kenya?

Again, I want to avoid a default of oversharing - I want to avoid moving the bar so that more PII is sent 1) than is currently sent and 2) than is necessary. I don't entirely know how to satisfy your use cases under that constraint, but the diversity of tax and shipping price practices across jurisdictions suggests that a fixed answer (e.g. send city and post code) will overshare for a non-trivial number of people.

@r12a r12a added the i18n-tracker Group bringing to attention of Internationalization, or tracked by i18n but not needing response. label Mar 18, 2019
@samuelweiler samuelweiler added the privacy-tracker Group bringing to attention of Privacy, or tracked by the Privacy Group but not needing response. label Mar 18, 2019
@ianbjacobs
Copy link
Collaborator

Hi @samuelweiler,

Thank you for the writeup.

According to the specs today, the merchant only receives billing address information through onpaymentmethodchange when requestBillingAddress is true. That information is redacted, as we've mentioned.

Thus, for some of the use cases you mention, where the merchant doesn't need any billing address information at all, that is the default behavior of the API. Merchants would explicitly need to set requestBillingAddress to true.

Ian

[1] https://w3c.github.io/payment-request/#dom-paymentoptions-requestbillingaddress

@ianbjacobs
Copy link
Collaborator

Re-opening as part of discussion to go to Proposed Rec.

@ianbjacobs ianbjacobs reopened this Sep 21, 2020
Payment Request 1.1 automation moved this from Done to In progress Sep 21, 2020
@ianbjacobs ianbjacobs added this to the Proposed Rec milestone Sep 21, 2020
@w3cbot w3cbot added privacy-needs-resolution Issue the Privacy Group has raised and looks for a response on. and removed privacy-tracker Group bringing to attention of Privacy, or tracked by the Privacy Group but not needing response. labels Sep 25, 2020
@pes10k
Copy link

pes10k commented Oct 12, 2020

Hi all, I was just wondering if there was an update on this issue, or if the WG could provide an update of where things stand. thanks much!

@EricMwobobia
Copy link

Great brainstorming on how to address the excess PII information . Any success in implementation? privacy wise this should be a great win for the end-users (customers paying via this feature).

@ianbjacobs
Copy link
Collaborator

Hi @pes10k,

I believe there has been no change to this issue. I am not aware of implementation experience for pull request #873, nor plans to implement. This was discussed briefly at the WPWG's 17 September call:
https://www.w3.org/2020/09/17-wpwg-minutes#item03

Ian

@Sauski
Copy link

Sauski commented Jan 14, 2021

I think it’s worth revisiting this, to see if the changes proposed in PR #873 (support for requesting the entire address directly however removed) can be included in the earlier version of the specification.

It feels wrong to introduce a spec that reduces how good a good actor can be over the current state. If a site previously made the decision to limit the address data it attempted to collect then we should be supporting that behavior and enabling it to continue.

In concert with this, absent any well defined, UA managed, incentives for sites to request less than the maximum possible information, it seems that the spec should aim to make the most private option the path of least resistance.

Introducing an intermediate state that is the combination of the least effort and least private option, then later attempting to persuade the ecosystem away from that position by offering a higher effort & higher privacy option, seems more challenging than simply avoiding offering the possibility in the first place.

@samuelweiler
Copy link
Member Author

@Sauski writes:

It feels wrong to introduce a spec that reduces how good a good actor can be over the current state.

The phrase "gratuitous harm to privacy" comes to mind. Thank you for giving this more voice, @Sauski.

@ricardo
Copy link

ricardo commented Feb 11, 2021

Hi everyone.

I thought I'd just add another use case here, because this seems to impact a lot of merchants using our WooCommerce Apple Pay implementation right now.

Currently, Apple Pay provides the merchants with only the first three characters of the ZIP code in the UK and Canada (e.g "SK1") when calculating the shipping methods. - The problem is that some merchants have wildcard postcodes (e.g "SK11*") with a minimum of 4 characters to display the available shipping methods. Redacting the postal code, in this case, breaks the checkout process for them.

It would be really nice to have the option to negotiate the postal code from being redacted.

@duerst
Copy link

duerst commented May 7, 2021

@marcoscaceres Any information on why this was closed?

@ianbjacobs
Copy link
Collaborator

Hi @duerst,

We removed support for addresses in the API (itself). Addresses may still be supported in private payment method data, but there are no longer any explicit API features for them.

@samuelweiler
Copy link
Member Author

... Addresses may still be supported in private payment method data, but there are no longer any explicit API features for them.

Pushing addresses out of the standard and into something not standardized is not a particularly satisfying answer - the privacy harm (potentially) remains the same, even though it has moved. That said, with it removed from this spec, I concur that this issue should be closed.

@ianbjacobs
Copy link
Collaborator

This issue was raised in Payment Request, but closed once we removed addresses from that API. We anticipate adding addresses back to the specification and so are re-opening this issue proactively.

@ianbjacobs ianbjacobs reopened this Feb 12, 2024
Payment Request 1.1 automation moved this from Done to In progress Feb 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Formal Objection Future Candidate Feature i18n-tracker Group bringing to attention of Internationalization, or tracked by i18n but not needing response. privacy-needs-resolution Issue the Privacy Group has raised and looks for a response on.
Projects
Payment Request 1.1
  
In progress