No-Vary-Search

Internet-Draft	No-Vary-Search	November 2024
Denicola & Roman	Expires 22 May 2025	[Page]

Status of This Memo

This Internet-Draft is submitted in full conformance with the provisions of BCP 78 and BCP 79.¶

Internet-Drafts are working documents of the Internet Engineering Task Force (IETF). Note that other groups may also distribute working documents as Internet-Drafts. The list of current Internet-Drafts is at https://datatracker.ietf.org/drafts/current/.¶

Internet-Drafts are draft documents valid for a maximum of six months and may be updated, replaced, or obsoleted by other documents at any time. It is inappropriate to use Internet-Drafts as reference material or to cite them other than as "work in progress."¶

This Internet-Draft will expire on 22 May 2025.¶

Copyright Notice

This document is subject to BCP 78 and the IETF Trust's Legal Provisions Relating to IETF Documents (https://trustee.ietf.org/license-info) in effect on the date of publication of this document. Please review these documents carefully, as they describe your rights and restrictions with respect to this document. Code Components extracted from this document must include Revised BSD License text as described in Section 4.e of the Trust Legal Provisions and are provided without warranty as described in the Revised BSD License.¶

1. Conventions and Definitions

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "NOT RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in BCP 14 [RFC2119] [RFC8174] when, and only when, they appear in all capitals, as shown here.¶

This document also adopts some conventions and notation typical in WHATWG and W3C usage, especially as it relates to algorithms. See [WHATWG-INFRA].¶

3. Data model

A URL search variance consists of the following:¶

no-vary params: either the special value wildcard or a list of strings¶
vary params: either the special value wildcard or a list of strings¶
vary on key order: a boolean¶

The default URL search variance is a URL search variance whose no-vary params is an empty list, vary params is wildcard, and vary on key order is true.¶

The obtain a URL search variance algorithm (Section 4.2) ensures that all URL search variances obey the following constraints:¶

vary params is a list if and only if the no-vary params is wildcard; and¶
no-vary params is a list if and only if the vary params is wildcard.¶

4. Parsing

4.1. Parse a URL search variance

To parse a URL search variance given value:¶

If value is null, then return the default URL search variance.¶
Let result be a new URL search variance.¶
Set result's vary on key order to true.¶
If value["key-order"] exists:¶
1. If value["key-order"] is not a boolean, then return the default URL search variance.¶
2. Set result's vary on key order to the boolean negation of value["key-order"].¶
If value["params"] exists:¶
1. If value["params"] is a boolean:¶
  1. If value["params"] is true, then:¶
    1. Set result's no-vary params to wildcard.¶
    2. Set result's vary params to the empty list.¶
  2. Otherwise:¶
    1. Set result's no-vary params to the empty list.¶
    2. Set result's vary params to wildcard.¶
2. Otherwise, if value["params"] is an array:¶
  1. If any item in value["params"] is not a string, then return the default URL search variance.¶
  2. Set result's no-vary params to the result of applying parse a key (Section 4.3) to each item in value["params"].¶
  3. Set result's vary params to wildcard.¶
3. Otherwise, return the default URL search variance.¶
If value["except"] exists:¶
1. If value["params"] is not true, then return the default URL search variance.¶
2. If value["except"] is not an array, then return the default URL search variance.¶
3. If any item in value["except"] is not a string, then return the default URL search variance.¶
4. Set result's vary params to the result of applying parse a key (Section 4.3) to each item in value["except"].¶
Return result.¶

4.2. Obtain a URL search variance

To obtain a URL search variance given a response response:¶

Let fieldValue be the result of getting a structured field value [FETCH] given `No-Vary-Search` and "dictionary" from response's header list.¶
Return the result of parsing a URL search variance (Section 4.1) given fieldValue. ¶

4.2.1. Examples

The following illustrates how various inputs are parsed, in terms of their impacting on the resulting no-vary params and vary params:¶

Table 1
Input	Result
`No-Vary-Search: params`	no-vary params: wildcard vary params: (empty list)
`No-Vary-Search: params=("a")`	no-vary params: « "`a`" » vary params: wildcard
`No-Vary-Search: params, except=("x")`	no-vary params: wildcard vary params: « "`x`" »

The following inputs are all invalid and will cause the default URL search variance to be returned:¶

No-Vary-Search: unknown-key¶
No-Vary-Search: key-order="not a boolean"¶
No-Vary-Search: params="not a boolean or inner list"¶
No-Vary-Search: params=(not-a-string)¶
No-Vary-Search: params=("a"), except=("x")¶
No-Vary-Search: params=(), except=()¶
No-Vary-Search: params=?0, except=("x")¶
No-Vary-Search: params, except=(not-a-string)¶
No-Vary-Search: params, except="not an inner list"¶
No-Vary-Search: params, except=?1¶
No-Vary-Search: except=("x")¶
No-Vary-Search: except=()¶

The following inputs are valid, but somewhat unconventional. They are shown alongside their more conventional form.¶

Table 2
Input	Conventional form
`No-Vary-Search: params=?1`	`No-Vary-Search: params`
`No-Vary-Search: key-order=?1`	`No-Vary-Search: key-order`
`No-Vary-Search: params, key-order, except=("x")`	`No-Vary-Search: key-order, params, except=("x")`
`No-Vary-Search: params=?0`	(omit the header)
`No-Vary-Search: params=()`	(omit the header)
`No-Vary-Search: key-order=?0`	(omit the header)

4.3. Parse a key

To parse a key given an ASCII string keyString:¶

Let keyBytes be the isomorphic encoding [WHATWG-INFRA] of keyString.¶
Replace any 0x2B (+) in keyBytes with 0x20 (SP).¶
Let keyBytesDecoded be the percent-decoding [WHATWG-URL] of keyBytes.¶
Let keyStringDecoded be the UTF-8 decoding without BOM [WHATWG-ENCODING] of keyBytesDecoded.¶
Return keyStringDecoded.¶

4.3.1. Examples

The parse a key algorithm allows encoding non-ASCII key strings in the ASCII structured header format, similar to how the application/x-www-form-urlencoded format [WHATWG-URL] allows encoding an entire entry list of keys and values in ASCII URL format. For example,¶

No-Vary-Search: params=("%C3%A9+%E6%B0%97")

will result in a URL search variance whose vary params are « "é 気" ». As explained in a later example, the canonicalization process during equivalence testing means this will treat as equivalent URL strings such as:¶

https://example.com/?é 気=1¶
https://example.com/?é+気=2¶
https://example.com/?%C3%A9%20気=3¶
https://example.com/?%C3%A9+%E6%B0%97=4¶

and so on, since they all are parsed [WHATWG-URL] to having the same key "é 気".¶

5. Comparing

Two URLs [WHATWG-URL] urlA and urlB are equivalent modulo search variance given a URL search variance searchVariance if the following algorithm returns true:¶

If the scheme, username, password, host, port, or path of urlA and urlB differ, then return false.¶
If searchVariance is equivalent to the default URL search variance, then:¶
1. If urlA's query equals urlB's query, then return true.¶
2. Return false.¶
In this case, even URL pairs that might appear the same after running the application/x-www-form-urlencoded parser [WHATWG-URL] on their queries, such as https://example.com/a and https://example.com/a?, or https://example.com/foo?a=b&&&c and https://example.com/foo?a=b&c=, will be treated as inequivalent.¶
Let searchParamsA and searchParamsB be empty lists.¶
If wrlA's query is not null, then set searchParamsA to the result of running the application/x-www-form-urlencoded parser [WHATWG-URL] given the isomorphic encoding [WHATWG-INFRA] of urlA's query.¶
If wrlB's query is not null, then set searchParamsB to the result of running the application/x-www-form-urlencoded parser [WHATWG-URL] given the isomorphic encoding [WHATWG-INFRA] of urlB's query.¶
If searchVariance's no-vary params is a list, then:¶
1. Set searchParamsA to a list containing those items pair in searchParamsA where searchVariance's no-vary params does not contain pair[0].¶
2. Set searchParamsB to a list containing those items pair in searchParamsB where searchVariance's no-vary params does not contain pair[0].¶
Otherwise, if searchVariance's vary params is a list, then:¶
1. Set searchParamsA to a list containing those items pair in searchParamsA where searchVariance's vary params contains pair[0].¶
2. Set searchParamsB to a list containing those items pair in searchParamsB where searchVariance's vary params contains pair[0].¶
If searchVariance's vary on key order is false, then:¶
1. Let keyLessThan be an algorithm taking as inputs two pairs (keyA, valueA) and (keyB, valueB), which returns whether keyA is code unit less than [WHATWG-INFRA] keyB.¶
2. Set searchParamsA to the result of sorting searchParamsA in ascending order with keyLessThan.¶
3. Set searchParamsB to the result of sorting searchParamsB in ascending order with keyLessThan.¶
If searchParamsA's size is not equal to searchParamsB's size, then return false.¶
Let i be 0.¶
While i < searchParamsA's size:¶
1. If searchParamsA[i][0] does not equal searchParamsB[i][0], then return false.¶
2. If searchParamsA[i][1] does not equal searchParamsB[i][1], then return false.¶
3. Set i to i + 1.¶
Return true.¶

5.1. Examples

Due to how the application/x-www-form-urlencoded parser canonicalizes query strings, there are some cases where query strings which do not appear obviously equivalent, will end up being treated as equivalent after parsing.¶

So, for example, given any non-default value for No-Vary-Search, such as No-Vary-Search: key-order, we will have the following equivalences:¶

https://example.com https://example.com/?: A null query is parsed the same as an empty string¶
https://example.com/?a=x https://example.com/?%61=%78: Parsing performs percent-decoding¶
https://example.com/?a=é https://example.com/?a=%C3%A9: Parsing performs percent-decoding¶
https://example.com/?a=%f6 https://example.com/?a=%ef%bf%bd: Both values are parsed as U+FFFD (�)¶
https://example.com/?a=x&&&& https://example.com/?a=x: Parsing splits on & and discards empty strings¶
https://example.com/?a= https://example.com/?a: Both parse as having an empty string value for a¶
https://example.com/?a=%20 https://example.com/?a=+ https://example.com/?a= &: + and %20 are both parsed as U+0020 SPACE¶

6. Caching

If a cache [HTTP-CACHING] implements this specification, the presented target URI requirement in Section 4 of [HTTP-CACHING] is replaced with:¶

one of the following:¶
- the presented target URI (Section 7.1 of [HTTP]) and that of the stored response match, or¶
- the presented target URI and that of the stored response are equivalent modulo search variance (Section 5), given the variance obtained (Section 4.2) from the stored response.¶

Servers SHOULD send no more than one distinct non-empty value for the No-Vary-Search field in response to requests for a given pathname.¶

Cache implementations MAY fail to reuse a stored response whose target URI matches only modulo URL search variance, if the cache has more recently stored a response which:¶

has a target URI which is equal to the presented target URI, excluding the query, and¶
has a non-empty value for the No-Vary-Search field, and¶
has a No-Vary-Search field value different from the stored response being considered for reuse.¶

Caches aren't required to reuse stored responses, generally. However, the above expressly empowers caches to, if it is advantageous for performance or other reasons, search a smaller number of stored responses. Such a cache might take steps like the following to identify a stored response (before checking the other conditions in Section 4 of [HTTP-CACHING]):¶

Let exactMatch be cache[presentedTargetURI]. If it is a stored response that can be reused, return it.¶
Let targetPath be presentedTargetURI, with query parameters removed.¶
Let lastNVS be mostRecentNVS[targetPath]. If it does not exist, return null.¶
Let simplifiedURL be the result of simplifying presentedTargetURI according to lastNVS (by removing query parameters which are not significant, and stable sorting parameters by key, if key order should be ignored).¶
Let nvsMatch be cache[simplifiedURL]. If it does not exist, return null. (It is assumed that this was written when storing in the cache, in addition to the exact URL.)¶
Let searchVariance be obtained (Section 4.2) from nvsMatch.¶
If nvsMatch's target URI and presentedTargetURI are not equivalent modulo search variance (Section 5) given searchVariance, then return null.¶
If nvsMatch is a stored response that can be reused, return it. Otherwise, return null.¶

Such implementations might "miss" some stored responses that could otherwise have been reused. It is therefore useful for servers to avoid sending different values for the No-Vary-Search field when possible.¶

7. Security Considerations

The main risk to be aware of is the impact of mismatched URLs. In particular, this could cause the user to see a response that was originally fetched from a URL different from the one displayed when they hovered a link, or the URL displayed in the URL bar.¶

However, since the impact is limited to query parameters, this does not cross the relevant security boundary, which is the origin [HTML]. (Or perhaps just the host, from the perspective of web browser security UI. [WHATWG-URL]) Indeed, we have already given origins complete control over how they present the (URL, reponse body) pair, including on the client side via technology such as history.replaceState() or service workers.¶

8. Privacy Considerations

This proposal is adjacent to the highly-privacy-relevant space of navigational tracking, which often uses query parameters to pass along user identifiers. However, we believe this proposal itself does not have privacy impacts. It does not interfere with existing navigational tracking mitigations, or any known future ones being contemplated. Indeed, if a page were to encode user identifiers in its URL, the only ability this proposal gives is to reduce such user tracking by preventing server processing of such user IDs (since the server is bypassed in favor of the cache). [NAV-TRACKING-MITIGATIONS]¶

9. IANA Considerations

IANA should do the following:¶

9.1. HTTP Field Names

Enter the following into the Hypertext Transfer Protocol (HTTP) Field Name Registry:¶

Field Name: No-Vary-Search¶
Status: permanent¶
Structured Type: Dictionary¶
Reference: this document¶
Comments: (none)¶

10. References

10.1. Normative References

[FETCH]: van Kesteren, A., "Fetch Living Standard", n.d., <https://fetch.spec.whatwg.org/>. WHATWG
[HTTP]: Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Semantics", STD 97, RFC 9110, DOI 10.17487/RFC9110, June 2022, <https://www.rfc-editor.org/rfc/rfc9110>.
[HTTP-CACHING]: Fielding, R., Ed., Nottingham, M., Ed., and J. Reschke, Ed., "HTTP Caching", STD 98, RFC 9111, DOI 10.17487/RFC9111, June 2022, <https://www.rfc-editor.org/rfc/rfc9111>.
[RFC2119]: Bradner, S., "Key words for use in RFCs to Indicate Requirement Levels", BCP 14, RFC 2119, DOI 10.17487/RFC2119, March 1997, <https://www.rfc-editor.org/rfc/rfc2119>.
[RFC8174]: Leiba, B., "Ambiguity of Uppercase vs Lowercase in RFC 2119 Key Words", BCP 14, RFC 8174, DOI 10.17487/RFC8174, May 2017, <https://www.rfc-editor.org/rfc/rfc8174>.
[STRUCTURED-FIELDS]: Nottingham, M. and P. Kamp, "Structured Field Values for HTTP", RFC 8941, DOI 10.17487/RFC8941, February 2021, <https://www.rfc-editor.org/rfc/rfc8941>.
[WHATWG-ENCODING]: van Kesteren, A., "Encoding Living Standard", n.d., <https://encoding.spec.whatwg.org/>. WHATWG
[WHATWG-INFRA]: van Kesteren, A. and D. Denicola, "Infra Living Standard", n.d., <https://infra.spec.whatwg.org/>. WHATWG
[WHATWG-URL]: van Kesteren, A., "URL Living Standard", n.d., <https://url.spec.whatwg.org/>. WHATWG

10.2. Informative References

[HTML]: van Kesteren, A., "HTML Living Standard", n.d., <https://html.spec.whatwg.org/>. WHATWG
[NAV-TRACKING-MITIGATIONS]: Snyder, P. and J. Yasskin, "Navigational-Tracking Mitigations", n.d., <https://privacycg.github.io/nav-tracking-mitigations/>. W3C Privacy CG

Acknowledgments

TODO acknowledge.¶

Authors' Addresses

Domenic Denicola

Google LLC

Email: d@domenic.me

Jeremy Roman