Performance Testing - Tools, Steps, and Best Practices
Web performance is a broad subject, and you'll find no shortage of performance testing tips and tutorials all over the web. Before you begin tuning your website or application, you must first figure out which metrics matter most to your users and establish some achievable benchmarks.
What is performance testing?
In the context of web development, performance testing entails using software tools to simulate how an application runs under specific circumstances. Quantitative performance testing looks at metrics like response time while qualitative testing is concerned with scalability, stability, and interoperability.
When the word "performance" is heard, most people immediately think of speed. Fast load times and response times are absolutely necessary these days, but you have to think about the bigger picture, which requires more than just clicking through all of your links to make sure they work. Just because everything works perfectly during production testing doesn't mean that will be the case when your website is flooded with traffic.
What are the benefits of performance testing?
Testing the performance of your website or application allows you to identify issues and improve the overall performance, which can lead to an improved user experience and increased revenue. There are many common issues that performance testing can uncover, such as bottlenecks. An interruption in data flow due to limited capacity is called a bottleneck. Bottlenecks can occur, for example, if you have a sudden surge in traffic that your servers are not equipped to handle. If you don't test, then you'll have to learn about them the hard way.
Bottlenecks are just one of many problems that can occur when your website isn't scalable. Poor scalability can cripple an application's performance resulting in delays, errors, and memory leaks. You may discover that your application's performance issues are due to CPU or bandwidth limitations, so you'll need to reallocate some resources or invest in a more robust infrastructure.
Any information you can collect about your visitors, such as how they are accessing your website, can help you identify ways to improve the user experience because you have the parameters to test against.
Types of performance tests
Although web page testing tools are helpful and can give you a quick overview of a website's speed and overall performance, you should conduct a more thorough analysis using a variety of tests including:
Load tests: Load tests look at how increased workload affects an application's response time. For example, you can use load testing tools to see how your application performs with a certain number of simultaneous users. The purpose of load testing is to evaluate how your application behaves under normal working conditions.
Stress tests: Also called fatigue tests, stress tests are similar to load tests, but they look at how an application performs outside the boundaries of normal working conditions. The goal of stress testing is to determine how many concurrent users or transactions the application can handle before it crashes. Load and stress tests can help you identify bottlenecks and decide how to best use your resources to accommodate more traffic.
Spike tests: A spike test is a specific type of stress test used to simulate application performance when the workload increases rapidly and repeatedly.
Endurance tests: Also called soak tests, endurance tests measure application performance over an extended period of time. Endurance tests can help you identify memory leaks and similar mishaps that occasionally occur.
Scalability tests: Scalability tests assess how well your application responds to increased workloads. Unlike spike tests, scalability tests involve gradually increasing workload while monitoring the effects on performance. You may also find that your resource usage fluctuates while the workload stays the same.
Volume tests: Also called flood tests, volume tests focus specifically on how your application performs while handling a large volume of data.
Which web performance metrics matter?
Improving your website's performance begins with taking some measurements, but measurements are not the same thing as metrics. What's the difference?
Measurements refer to specific data points, such as the number of seconds it takes to process a request. Metrics are what is actually being measured. Here are some metrics relevant to performance testing:
Response time: The amount of time between a specific request and a corresponding response. Response times can vary drastically for different actions under different conditions.
Average load time: The average response time for all requests gives good insight into the overall user experience.
Peak response time: The longest response time. If your peak response is much longer than your average load time, then you probably have a problem.
Wait time: Sometimes called average latency, wait time refers the amount of time a request spends in a queue before it gets processed. Distinguishing between wait time and response time is important because they depend on different factors.
Requests per second: The number of requests handled per second.
Memory utilization: The amount of memory required to process a request.
CPU utilization: The amount of time required for the CPU to process requests.
Error rate: The ratio of errors to requests.
Transactions passed/failed: Similar to error rate, but it takes into account other factors that cause requests to fail.
Concurrent users: Also called the load size, concurrent users means the number of active users.
Throughput: Typically measured in kilobytes per second, throughput refers to how much bandwidth gets used during performance testing. Since it is the best indicator of your website's capacity, setting a throughput goal is a good first step to improving your website's capacity.
Performance testing tools for web developers
The first step to testing is selecting the right tools. There are many free and premium tools to test your apps, but here are just a few:
KeyCDN Tools
In addition to our content delivery services, KeyCDN Tools is a set of free web testing tools that we publicly offer. Our page speed tests gives you a full waterfall breakdown of how your website performs, and you can test how fast your page loads from 10 different locations.
Google Lighthouse
Google Lighthouse is a free and open source tool that is part of the Google Chrome DevTools family. When a URL is provided to Lighthouse, it will run several audits and return a report with advice for improvements.
Selenium
Selenium is a lightweight, open source testing framework for recording, editing, and debugging tests.
Loadstorm
Loadstorm is an affordable premium option for load testing that can test up to 50,000 concurrent users. The cloud-based service sends massive numbers of requests per second to simulate how your website holds up.
WebLOAD
WebLOAD offers a robust toolset that is top of the line. It is tailored to enterprise applications that have thousands of users, but limited free edition is available.
Performance testing step by step
Once you've settled on which tools to use, here is a general guide to follow as you test your website's performance:
- Set goals: Decide which metrics matter most to your users and establish some ideal benchmarks.
- Plan your test scenarios: Come up with some specific situations your application is likely to encounter. For example, what would happen if a certain number of users try to perform a certain action at the same time?
- Prepare the testing environment: Make sure your hardware and network configurations closely reflect real world conditions.
- Test and collect data: Let your testing software do all the hard work, then assess the results.
- Repeat: Run the tests under the same conditions again to ensure that performance is consistent, then experiment with different parameters.
Once you quantify your website's performance with data, you can start taking steps to resolve any issues identified.
More performance testing tips
Test early and test often. The earlier you implement testing in an application's life cycle, the easier it will be to address problems as they arise. Start during the unit testing phase if possible. It is important to be realistic. Your website or application's performance will be limited by your resources. Know your limits so that you can set achievable goals. Create realistic test scenarios; for example, don't start your performance tests at zero load since that is an unrealistic situation.
Like everything else in web development, your web performance plan depends on your audience. If you find you have a lot of users from other countries, then you need to make sure you have the infrastructure to support international visitors. If most of your visitors are using their phones, then you need to focus on optimizing for mobile devices. Many testing tools let you monitor user behavior in great detail.
Take the time to understand and know your testing environment. Using a test environment that is similar to the production environment is essential to getting an accurate picture of web performance. Use the same environment if possible.
Consider optimizing before investing. Buying new servers and more bandwidth without optimizing your current resource is a waste of money. Focus on making your infrastructure more efficient before you even think about investing in more.
Performance testing - A job that never ends
Performance testing should be an on going process. Hopefully, your website or application will continue to grow, and you'll need to make changes to accommodate a larger user base. If you fail to test, you could become a victim of your own success.