KeyCDN Enables HTTP/2 HPACK Compression - Huffman Encoding

KeyCDN is excited to announce that we have enabled HPACK compression, which utilizes Huffman encoding, for all of you who are running over HTTPS on HTTP/2. This is now running on all of our edge servers and there is no configuration necessary on your part. HPACK compression reduces the size of your headers by over 30% on average. There is less data now being sent which in turn will increase your content delivery speeds. Read more below about how HPACK and Huffman encoding allow for further compression of binary data in your headers.

HTTP/2

As most of you know HTTP/2 is a new protocol based off of SPDY and is the first major protocol update since HTTP/1.1. KeyCDN launched support for HTTP/2 back in October 2015, making us one of the first CDN providers to do so. Curious on the growth rate of HTTP/2 among KeyCDN users? We were too so we pulled some data to show you. The chart below reveals the distribution of the HTTP/2 traffic in percentage to the total HTTPS traffic delivered by KeyCDN. Back in October it was just passing the 50% mark.

And as of April 13th, 2016, HTTP/2 traffic has now reached 68%. So in 6 months there has been a 17% increase in HTTP/2 traffic and adoption.

If you are interested in seeing more granular data at a country level, we have just updated our post on HTTP/2 statistics, make sure to check it out!

HPACK Compression - Huffman encoding

There are a lot of performance benefits with HTTP/2 and one of the most exciting ones is definitely HPACK compression. Unlike HTTP/1.1, headers can now be compressed using an algorithm known as Huffman encoding which in turn reduces the amount of data being sent. In our tests we showed it resulted in a decrease of header sizes by an average of 30%. Which is pretty significant! There are three main benefits to HPACK:

• Ability to encode large headers using fixed Huffman encoding.
• Ability to encode commonly used headers as a variable length integer, rather than re-sending the whole header each time (compressing strings).
• Because of encoding, it is not vulnerable to compression based attacks such as CRIME.

All headers sent via HTTP/2 (in HEADERS, CONTINUATION and PUSH_PROMISE frames) are sent in the HPACK format. And remember HTTPS is not technically required for HTTP/2 but browsers only currently support it via an encrypted connection (TLS).

Basically the way Huffman encoding works is that it assigns binary codes to frequently used characters. When someone receives a Huffman encoded header it looks them up in the list of binary codes. In the end the header is using frequently used characters and in turn takes less bits to transmit. For example, in the string accept we would count the frequencies of each character and generate a code (with the shortest codes going to the most frequent):

b - 0
a - 101
e - 111
m - 110
s - 100

Then, to transmit accept we just replace each character with its code:

101 0 0 111 110 100

The original string encoded with 7 bits per character would take 6 bytes. The new Huffman encoded version only takes 2 bytes and therefore requires less data to be transferred in the header.

Another advantage of HPACK is that it uses indexing. The table contains frequently used headers like User-Agent. If a header is sent that exists in the table the index is used instead of the literal string.

HTTP/2 vs HTTP/1.1

It is also important to understand that HTTP/2 uses differential encoding. Headers are now stored as explicit key value pairs. On the first request, a full compliment of headers are sent. On subsequent requests, only the difference from the headers in the first request are sent. This is much more efficient than that of an HTTP/1.1 request. Here is an example below. Let's say you need to fetch two objects, /index.html and /logo.svg.

First request - HTTP/1.1

Here is the first request sent via HTTP/1.1:

GET /index.html HTTP/1.1
Host: www.keycdn.com
Referer: https://www.keycdn.com/
Accept-Encoding: gzip

First request - HTTP/2

And here is the first request sent via HTTP/2:

:method: GET
:scheme: https
:host: www.keycdn.com
:path: /index.html
referer: https://www.keycdn.com/
accept-encoding: gzip

Second Request - HTTP/1.1

Here is the second request via HTTP/1.1. Notice how it must pass the host and encoding again:

GET /logo.svg HTTP/1.1
Host: www.keycdn.com
Referer:https://www.keycdn.com/index.html
Accept-Encoding:gzip

Second request - HTTP/2

Here is the second request via HTTP/2. Since the method, host, and encoding values are the same they are no longer passed in the subsequent request:

:path: /logo.svg
referer: https://www.keycdn.com/index.html

And that is just the header request. HTTP/2 also offers the following advantages over HTTP/1.1:

• It is fully multiplexed, instead of ordered and blocking.
• It can use one connection for parallelism.
• It allows servers to "push" responses proactively into client caches instead of waiting for a new request for each resource.

And as you now know, it also uses much better header compression and is binary instead of textual.

Summary

As you can see HTTP/2 HPACK compression, along with Huffman encoding allow us to speed up content delivery even more by decreasing the overall sizes of your headers. Is your site or CDN provider running over HTTP/2 yet? Remember, you can always check using our HTTP/2 test tool. There has never been a better time to migrate to HTTPS and take full advantage of these new features.