Measuring Web Performance - Analyzing What Matters Most
Professional web performance management has become its own industry, and rightfully so, as performance is vital to improving conversion rates, user retention, and more. However, the process of measuring web performance can take many shapes and with all the web performance metrics available it can be confusing to know which ones you should be tracking. This guide will serve as an overview of the ways you should be measuring web performance as well as provide additional insight and tools to help improve your site's web performance.
How do you measure web performance?
Since there are dozens of metrics that relate to performance, you need to choose which ones are most relevant to your users. Below are some of the most common indicators that developers look at when measuring web performance:
Page load
Page load is usually defined as the amount of time it takes for the browser to fire the window.onload JavaScript event handler. Ideally, you'll want your page load time to be less than 2 seconds.
If it's more than this you should look into implementing web performance optimization techniques. Consequently, it has become common practice to delay the loading of certain elements, such as advertisements and third party scripts, until after the onload event has executed. This allows the important assets to load first and then start loading any third party scripts.
Render time
Render time may be defined in a couple of ways. One benchmark to consider is the DOMContentLoaded event, which fires once the initial HTML document has been loaded and parsed. Perhaps a more meaningful metric to look at is time-to-first-paint, or TTFP. TTFP is how long users have to wait until they see something besides a blank browser display.
Comparing your page load times to your render times can provide important insights. For instance, slow render times could be indicative of an issue with the JavaScript or CSS files in your render pipeline. Check out our complete guide on what's blocking the DOM? to learn how you can identify such bottlenecks. Focusing on both of these metrics can simultaneously improve the user experience and your search engine rankings.
Time to first meaningful paint
The "first paint" metric tells us very limited information about how webpages load. That's why we now also measure "the time to first meaningful paint," or the point when users actually see a meaningful image. Of course, it's up to developers to decide what's "meaningful." Take advantage of directives such as preload to improve the load times of critical assets.
Server time
Server time measures how long the server takes to respond to an initial request. It's similar to time-to-first-byte, or TTFB, which is the amount of time it takes for the user to receive the first byte of data related to an HTTP request. Since the base HTML file is always the first thing that users receive, TTFB is also a good indicator of your backend processing time.
Webpages and apps that rely heavily on databases typically have higher server times, which is why configuration optimization is so important. If you're a WordPress user and want to easily reduce your TTFB, use a caching plugin such as Cache Enabler.
Speed index
Since its introduction to WebPageTest.org back in 2012, the Google speed index has become a common standard for measuring web performance. According to Google, speed index is the average amount of time it takes for all of the visual elements in the viewport to display. Therefore, it is dependent on the viewport size.
WebPageTest calculates the speed index in milliseconds by capturing video of a webpage as it loads. Your speed index can tell you more about the user experience than TTFB since it measures how quickly the user actually sees content. Since the speed index is based on a percentage of the viewport, it's a useful metric to consider when optimizing your app for different screen sizes.
Bytes in
Another metric specific to WebPageTest, "bytes in" is just Google's fancy way of saying page size. It refers to the number of kilobytes required to download all of a page's content. Page sizes tend to increase over the years, which can be a major problem for some users since large webpages take longer to load over mobile networks.
Now that a large part of internet traffic comes from mobile users, and since 40 percent of mobile users give up on websites that take longer than 3 seconds to load, it's a good idea to keep your pages as small as possible. Optimizing your images and minifying your JavaScript and CSS files can help to that end.
Cost
Cost is another WebPageTest exclusive metric that estimates the monetary cost to users when they view your website on mobile devices from different parts of the world.
Knowing this information gives you yet another incentive to keep your page sizes down.
Time to complete a transaction
Calculating how long it takes for users to complete a financial transaction is difficult because the process typically involves several steps across multiple pages. Fortunately, monitoring services like Site24x7 let developers conduct multiple-step simulations for such purposes.
Time to interact
Before webpages can even finish loading, users typically start scrolling and clicking. Picking out the point when visitors start engaging with your application can give you further insight into the user experience. This can be accomplished by adding specific JavaScript events and handlers. If you notice any "dead spots" on your webpages, you might want to consider getting rid of elements that receive little or no attention from users.
Why measure web performance?
Measuring web performance isn't just about preserving the status quo. Monitoring metrics allows you to uncover new ways to improve your applications. However, before you can do that, you need a solid understanding of the relationship between your metrics and the user experience. Figuring out that relationship might require some creativity. For example, the Financial Times figured out that a single second increase in load time resulted in nearly a 5 percent drop in article views.
Server time, render time and page load are relevant to all apps, but beyond those metrics, you'll have to decide what's most important to your users. Just remember that web performance management always entails trade-offs, so whenever you make a change for the sake of one metric, always check to see how it impacts anything else.
Defining performance with a KPI
You can't measure something that you can't define, which is where a key performance indicator, or KPI, can come in handy. A KPI is a single, measurable value that reflects your progress toward a specific performance goal. If you're part of a larger development team, everyone should be familiar with the KPI so that they can take it into account when making important decisions.
For example, let's say your KPI is the amount of time it takes for text to appear on your homepage, and you choose a target of 1.7 seconds. If someone wants to suggest adding more advertisements to the homepage, they should first find out exactly how much time it would add to the KPI. Is adding a few milliseconds to your load time worth the potential extra revenue? Such questions aren't always easy to answer, but having a KPI gives you a starting point to weigh the pros and cons.
Of course, when selecting a value for your KPI, you must consider your testing environment.
Is your goal of 1.7 seconds geared toward desktop users in the U.S. or mobile users in Uganda? Either choice is fine, just keep it in mind when making business decisions.
Making a performance budget
Similar to KPIs, performance budgets are numbers that team members agree upon to guide the development process. Your performance budget could include statements like, "Images should account for less than 30 KB of the total page size." Use the performance budget calculator to see what your performance budget is given the load time you desire and connection speed you define.
Keeping an eye on your page sizes can help you identify redundant resources and unnecessarily long scripts. Check out our complete guide on Setting and Calculating a Web Performance Budget.
Real user monitoring vs synthetic testing
One of the biggest challenges to measuring web performance is anticipating how users will access your application. Where they live, which device they use and which browser they run are just a few of the factors that impact individual users' experiences. Ideally, your application should work just as well for everyone all the time, but that's rarely the reality. Most developers have limited resources, so they must decide how to allocate them. Therefore, user data is perhaps one of the most important things to look at when measuring web performance.
Real user monitoring, or RUM, means collecting data from users in real time rather than running simulations. Such information is more accurate than data collected from synthetic testing, but it's limited to your current visitors, so a combination of both types of testing is most valuable. New Relic and Pingdom offer premium RUM services, but you can set up your own light version of real user monitoring using analytics services such as Google Analytics.
Tools and resources for measuring web performance
You can find plenty of premium products for managing and measuring web performance, but there are also several free resources available online. We've actually compiled a list of the top 15 speed test tools of 2017. Each tool is slightly different however they each provide valuable information pertaining to the loading times of your website.
If you're looking for a lightweight tool with multiple locations to test from check out KeyCDN's speed test tool.
Furthermore, a great resource from Google is Lighthouse, which was created specifically for testing progressive web apps. In addition to tools like Lighthouse and WebPageTest, Google offers extensive advice for improving web performance in their DevTools Docs.
Summary
As your projects grow, you should periodically reassess your performance management plans to determine if the benchmarks and metrics you're tracking are still relevant to your users. For example, if you notice that you have more mobile users than you originally anticipated, you may want to direct more resources to monitoring and improving mobile performance. Regardless of who you're optimizing for, measuring web performance and knowing what needs to be optimized is the first step to having a fast loading website.