Why You Should Load Test with Real Browsers

Imagine your software-as-a-service company is converting a desktop application into a web application. In addition to a JavaScript-heavy front-end, the web application interacts with numerous third-party services, such as content delivery networks and analytics providers. Unit and integration tests are passing, but you want to ensure that the application scales to thousands of concurrent users before deploying it live.

You could use a load testing tool, like Apache JMeter, to see how the application performs under load. While it requires extensive scripting for sessions and cookies, you will see exactly how the application (on the server-side) would respond to large numbers of requests. The problem is that these protocol-based tests don't measure the performance of on-page HTML, JavaScript and other elements that impact the user experience (on the front-end).

Let's take a look at how protocol-based load testing works and why real browser testing may be a superior approach — especially when working with front-end heavy applications.

Protocol-based load tests are time-consuming to create and don't provide a complete picture of performance.

Load Testing 101

Performance testing is an important part of the software development lifecycle. By simulating real users, test engineers can see how the software behaves in a variety of scenarios. Unit tests ensure that code produces a desirable outcome and integration tests ensure that the pieces fit together, but neither ensures that the software will perform under stress.

For instance, an airline might be planning a promotion that will result in a significant increase in traffic. Load tests can help ensure that the booking engine can handle high levels of concurrent users without experiencing errors. After all, the last thing the airline wants is to spend money on a promotion only to disappoint would-be buyers with a blank page!

Load testing identifies bottlenecks before deployment and reduces the risk of downtime during peak times. For example, developers may notice that expensive database queries cause availability issues at a certain threshold. These defects are hard to notice without load testing and may only become apparent when heavy traffic occurs in production.

Real vs. Simulated Browsers

Most load tests for web applications simulate traffic on the protocol level. For example, Apache's JMeter looks like a browser to a server, but it's not an actual browser instance. It assesses the performance of the network and server, but doesn't actually render HTML or execute JavaScript, which means that it may not match the actual user experience.

Check out our free Checklist of Load Testing Metrics to Watch to ensure you're looking at the right data.

Protocol-based test frameworks also require extensive scripting to accurately measure performance. Test engineers must record requests in Developer Tools and customize dynamic requests that contain sessions or cookies. In many cases, development teams outsource performance testing to third parties due to the high level of complexity.

These two limitations create a few problems:

  • Single-page applications (SPAs), third-party scripts (e.g. analytics or advertisements) and other JavaScript can have a significant impact on load time. Protocol-based testing frameworks don't include page render time in any of their calculations — only page duration (e.g. HTTP requests). The differences in rendering times can vary.
     
  • Protocol-based frameworks require you to write hard-to-read scripts that need dynamic correlation of session stage for simple playback. For example, a test engineer must look at the requests made in Developer Tools and translate them into a load test script, while accounting for any sessions or cookies that may require unique details.
     
  • It's impossible to test specific device and screen resolution performance. For example, a responsive website may have very different performance characteristics for different resolutions. It's hard to diagnose performance that impacts these specific devices using protocol-based tests — and in some cases, they may be missed entirely! 
How to Use Real Browsers

Browser-based load testing is challenging without the right tools. After all, running a browser is very resource-intensive and each browser instance requires its own CPU core. Simulating just 1,000 real browsers would require 200 different high-performance computers, which is a lot of resources even given today's competitive cloud-computing environment.

LoadNinja makes it easy to run load tests with real browsers. Rather than recording HTTP/S protocol traffic to create test scripts, you can use real browsers in the cloud to record and playback tests without any dynamic correlation. The simplified approach alone cuts down on load testing time by half and represents a breakthrough in performance testing.

Old-vs-New-Way.png

In addition to saving time, LoadNinja lets you generate a load using tens of thousands of real browsers to provide the most realistic and accurate performance data for your web application. You can even test different screen resolutions or focus on a specific size to identify resolution or device-specific issues that are hard to troubleshoot.

Finally, LoadNinja breaks down the results of these tests to help you rapidly debug performance issues. You can analyze web application performance using navigation timings and other data sourced straight from the browser. With the VU Inspector and VU Debugger, you can inspect and debug performance issues in real-time without having to dig deep to find bottlenecks.

The Bottom Line

Load testing is an important part of the software development lifecycle. While protocol-based tests have become the standard for load testing, they require extensive scripting time and knowledge, and they don't provide the most accurate picture of the user experience. The only way to do that is with real browsers that render the entire page as a user would.

Don't forget to check out our free Checklist of Load Testing Metrics to Watch to ensure you're looking at the right data.

LoadNinja simplifies the load testing process and makes it economical to load test with real browsers in the cloud. You can even integrate these tests with your existing CI/CD processes to bring performance testing in-house and ensure that your application is always performant before being deployed.

Sign up for a free trial of LoadNinja today to discover how easy load testing can be (with real browsers).