Vary Header

One of the most important HTTP headers is the Vary header. Depending on your setup, this must be configured to ensure the right content is delivered to the right browser. When requesting content, browsers include HTTP headers to let the server decide what to send back, whether it's for a mobile client, a browser that can or can't handle Gzip, or a browser requesting a certain language. Here is what an average HTTP request looks like.

GET /foobar.php HTTP/1.1
Host: example.com
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8
Accept-Encoding: gzip,deflate,sdch
Accept-Language: en-US,en;q=0.8
Connection: keep-alive
User-Agent: Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125Safari/537.36

These HTTP headers work great when requests are going directly to the origin server and back to the browser, however, things get a little tricky when caching and proxies are involved. The Vary header helps solve the confusion which will be discussed in greater detail in the following sections.

The issue

Imagine there are two browsers, one browser doesn't understand compression requests and one does. Between the client and the caching server, there is a proxy server that is also caching assets and determining what to send which browsers. The browser that requests a new page first will determine whether that page will be stored in the proxy as a compressed or uncompressed version. The problem is if the older browser that doesn't understand compression requests the page first, then the uncompressed version of that page will be stored in the proxy. Therefore, when the modern browser goes and requests the same page, it will be served that uncompressed version from the proxy, resulting in a longer load time.

On the other hand, what's worse is if the newer browser that does understand compression requests the page first, the compressed version of that page will be stored in the proxy. When the older browser then visits that page, it will be served the compressed version that it will not be able to understand and, therefore, will be unreadable to the user.

Based on the above example, this reason is why the Vary header is extremely important in today's age of mixed browser versions. According to RFC 2616, section 14.44: Vary Header,

An HTTP/1.1 server SHOULD include a Vary header field with any cacheable response that is subject to server-driven negotiation. Doing so allows a cache to properly interpret future requests on that resource and informs the user agent about the presence of negotiation on that resource.

Vary header

The Vary HTTP header fixes the old vs. new browser dilemma by telling the HTTP cache what parts of the request header to use when trying to find the right object. It does this by adding the names of relevant headers, which, in the case of the compression example, would be Vary: Accept-Encoding. An example of an uncompressed response header would look something like this.

HTTP/1.1 200 OK
Content-Type: application/javascript
Content-Length: 5135
Connection: keep-alive
Cache-Control: max-age=604800
Vary: Accept-Encoding

An example of a compressed response header would look like (note that this example includes Content-Encoding: gzip):

HTTP/1.1 200 OK
Content-Type: application/javascript
Content-Length: 5135
Connection: keep-alive
Cache-Control: max-age=604800
Content-Encoding: gzip
Vary: Accept-Encoding

The Vary HTTP header is present whether the server is responding with a compressed or uncompressed version of an object. This is due to the fact that after the browser requests an uncompressed version of an object from the server, that object would then get stored in cache with a flag that says, "This object should only be served to requests that don't have Accept-Encoding in the request".

Once the older browser has already requested this object, let's say a newer browser that does understand compression requests the same object. The newer browser, therefore, contains the HTTP header Accept-Encoding with values set to gzip, deflate, sdch. The object is already in the cache, however due to the flag that was set, this object cannot be served to the newer browser as it contains Accept-Encoding in the request. The request goes back to the server to retrieve the compressed version which is now stored in the cache with a flag that says, "This object should only be served to requests that have Accept-Encoding: gzip, deflate, sdch" in the request.

Vary header and SPDY / HTTP/2

SPDY is an open networking protocol developed for the transportation of web content. It was developed with the end goal in mind of reducing latency, and improving website security. HTTP/2 is based on SPDY. However, HTTP/2 uses a different compression algorithm than SPDY, which helps make the protocol less vulnerable to attacks. SPDY and HTTP/2 were implemented because the HTTP/1.1 protocol was quite simply starting to show it's age in today's online world. The key differences between HTTP/1.1 and HTTP/2 include the following:

• HTTP/2 is binary rather than textual
• It is fully multiplexed, instead of ordered and blocking
• It can use one connection for parallelism
• It uses header compression
• It allows servers to "push" responses proactively into client caches

In the example in the section above, the Vary header helps determine what type of object (compressed or uncompressed) is delivered to the browser based on compatibility. However, SPDY and HTTP/2 have their own dedicated header compression mechanism. The standards for SPDY / HTTP/2 imply that the web browser has to support data compression.

As stated in section 3.2.1 of the Chromium SPDY protocol guide,

User-agents MUST support gzip compression. Regardless of the Accept-Encoding sent by the user-agent, the server may always send content encoded with gzip or deflate encoding.

SPDY decided that this rendered the Vary: Accept-Encoding header useless, as a result, omitted it from the HTTP header section. HTTP/2 however has since brought back the addition of the Vary header and it is now present in response headers just as it was in HTTP/1.

Adding the Vary HTTP header to your origin server

If you're using Apache's mod_deflate, the correct Vary HTTP header is automatically added to your responses. However, if you aren't using it, here's how to add Vary: Accept-Encoding to your origin server.

Apache

# Check if the headers module is active
<IfModule mod_headers.c>
    # match certain file types that are compressible and add the vary header if yes
    <FilesMatch ".(html|css|xml|js|json|svg)$">
        Header append Vary "Accept-Encoding"
    </FilesMatch>
</IfModule>

Nginx

# directive to enable Vary: Accept-Encoding
gzip_vary on;

IIS

<system.webServer>
    <httpProtocol>
        <customHeaders>
            <remove name="Vary"></remove>
            <add name="Vary" value="Accept-Encoding"></add>
        </customHeaders>
    </httpProtocol>
</system.webServer>

Summary

Based on the example outlined in this post, the importance of the Vary header is evident. Users who are still using old browser or versions that do not support Gzip compression are at high risk of receiving a response that is unreadable without use of the Vary header. However, with the Vary header, determining which browser should receive compressed/uncompressed content versions can be easily achieved.