How to Effectively Monitor and Scale for Website Traffic

Scaling when it pertains to web performance essentially means the ability to support any traffic load. Unfortunately most business owners don't think about this until their website goes down due to heavy traffic. That is why it is very important to plan for scaling and try to anticipate your peak load in advance with performance tools such as real user monitoring, server monitoring, and load testing. Once you know where your load is at you can put measures in place to avoid downtime and frustrated visitors down the road.

When do you need to scale?

The first question you are probably wondering is, when do you need to scale? Another way to look at scaling is essentially you are wanting to keep the behavior of the application or website the same as the load (traffic or number of users) increases. If your website loads in under 2 seconds right now with 500,000 visitors a month, then you still want it to load under 2 seconds once you cross 1 million visitors a month. Another important factor besides total website traffic is the number of concurrent users you are averaging. Most of the time you can scale simply by adding resources such as additional bandwidth, CPU, and memory, but that is not always the case based on the complexity of your infrastructure and or website.

The answer is you should start planning out your scaling strategy now

58% of application/site owners overestimate their capacity levels.

- State of Web Readiness Report

Here are some interesting performance statistics from a case study performed by Load Storm.

• 68% of all survey respondents experienced performance or stability problems with their website last year. Lack of resources was the main reason for these problems, both among ecommerce sites and other websites.
• 39% of e-retailers claimed they lost money last year due to performance or stability problems. In comparison, only 24 percent of the non-e-retailer respondents claimed they lost money.
• 98% of e-retailers thought sub-two-second response time was desirable. However, load time for ecommerce sites analyzed exceeded this response threshold by a wide margin. The average was 7.9 seconds - higher than for non-ecommerce sites

It is typical for 90% of your website's response time to be from frontend resources, but this changes as your load and traffic increases.

When you are determining what you need to do when scaling it is good to ask yourself these following questions:

• Does the website or application perform acceptably with a minimum amount of users on the system?
• Does the website or application respond quickly with additional concurrent users? And if so how many?
• Are there hardware bottlenecks in the current system?
• Can the website or application handle a growing number of users/data?
• What is the maximum amount of load the system can handle?

Why scaling is important

Scaling is very important because you never know when that spike of traffic is going to happen. Having a plan in place will prevent disasters such as Target's website going down on Cyber Monday due to the overwhelming amount of traffic to its website. Shoppers were met with a "please hold tight" message Monday morning on Target.com.

The traffic, the demand, the orders are just absolutely through the roof.

- Katie Boylan, Director of PR at Target

During the holidays more and more retailers are seeing people stay home as consumers choose to shop online to avoid the rush. This means online retailers need to focus on scaling so they can handle the loads.

Scale testing tools

Below are tools you can use to pinpoint where your server, website, or application is having performance issues such as CPU bottlenecking, timeout errors, high network utilization, etc.

Real user monitoring

RUM, also known as, real user monitoring, is an approach to capturing and analyzing every user interaction on your website. It is a form of passive monitoring that determines whether or not users are being served quickly. If the system determines there is a lag in service, it will pinpoint the problem area so that operators may quickly rectify the issue. Here are a couple popular RUM tools you can use to gain performance insights on user interactions:

• Google Analytics: Free basic RUM tool that exists in your GA dashboard under the "Behavior" menu.
• Pingdom: Insight into performance and helps you answer questions like, how does your website really perform from a specific country or web browser.
• New Relic: Historical context into the experience actual users have with your browser-side applications.

Server monitoring and application performance management

Analyze server processes and application performance with some of these popular tools:

• New Relic: Quickly pinpoint if it was your code, backend server, or 3rd-party service that caused an issue.
• AppDynamics: Boost application performance by resolving server bottlenecks quickly.
• SolarWinds: Identify how applications performance is impacted by underlying infrastructure.

Load testing

Load testing tools can an easy way to throw thousands of concurrent users against your website or application and see how it holds up. Here are some popular load testing tools:

• CyberFlood: Security and performance testing on websites and applications.
• Load Impact: Automated and on demand performance testing for DevOps. Load test your website, web app, mobile app or API instantly with up to 1.2 million concurrent users.
• Loader: Load testing service that allows you to stress test your web-apps & APIs with thousands of concurrent connections. Free up to 10,000 clients.
• BlazeMeter: Run massively scalable, open source-based performance tests against all of your apps, and validate performance at every software delivery stage.

Some of these tools also allow to you install a local server agent which will show you additional useful backend insights such as:

• CPU usage
• Memory usage
• APDEX
• Disk I/O
• Network I/O

Example load test

We ran an example load test with Loader on our case study site. We ran it with 5,000 users over the course of a minute, or around 80 concurrent users per second. This is a small 1 CPU, 1024 MB VPS. Loader is really simple to setup a quick test. See our steps below.

Step 1

Signup for an account with Loader. (free up to 10,000 users). Input the domain you want to test.

Step 2

Confirm your domain by either uploading a file via FTP or via DNS.

Step 3

Configure your load test. As you can see we did a simple test with 5,000 users over the course of a minute.

We then started the test and it went fine up to around 27 seconds where it started getting timeout errors from the VPS.

If we dig a little more into the logs I can see the CPU utilization jumped to 81% (which was probably the problem) and the memory usage was at 55%. We could dig into the Nginx logs a bit more to really pinpoint the issue, but we already knew at this point our VPS couldn't handle this many concurrent users.

So the next step would be to add more resources, scaling our VPS, and re-run the test. This was just an example. For proper testing it is usually recommended you run a baseline test, then a load test, and then a stress test.

How a content delivery network (CDN) can help

A CDN can be a cost-effective way to easily help you scale your environment by letting it deliver all of your static as well as dynamic assets. Some people have been able to decrease their bandwidth utilization and total requests by 90%. This allows your origin server to handle more load than it would stand alone.

A couple quick reasons why you might want to consider a CDN.

• Your site is growing at an exponential rate - A normal server is going to have a tough time handling a spike in traffic, and a CDN excels at this by distributing all the requests across different edge servers all over the globe, decreasing the chance of your website going down dramatically. Many websites have crashed after going on shows such as Shark Tank and it was because they didn't prepare for the event and their infrastructure couldn't scale with the website traffic.
• Large files and static content are slowing your site down - Items like large images, scripts, CSS and other elements can prevent your site from loading quickly. Especially for online magazines and sites that use tons of media.
• Your audience is starting to expand worldwide - If a lot of your visitors are starting to come from different countries then it is definitely time to look at a CDN so that your website will load faster in their physical location. At that point it all comes down to latency and the distance from the nearest POP.

Other website traffic scaling methods

Here are some additional ways to scale out your website that you will want to keep in mind:

• Load balancing
• High-level caching
• Bigger and faster servers with more resources (e.g. CPU and memory)
• Faster disks (e.g. SSDs)
• Scalable databases
• Bandwidth/Network upgrades

Summary

As you can see scaling your infrastructure to meet the demands of your website traffic, no matter what the spike might be, can be a complicated and time-consuming process. It usually involves load testing, user monitoring, application monitoring and lots of planning. If you don't currently have a scaling plan in place, we highly recommend you get to it!

What has been your experience? Has your website gone down due to a traffic spike? Let us know about it in the comment section below.