User experience is an important brand differentiator and poor experiences can be costly to brand reputation — load testing can ensure a great experience.
 

Protocol vs. Browser Tests

Load testing involves generating artificial “users” of an application and measuring its response. You can easily measure response times, throughput rates, and resource utilization levels before code changes reach production users. That way, you can ensure a consistent user experience, avoid any costly performance issues, and meet service level agreements.

While conventional load tests have occurred right before a deployment (or even after), Agile development teams are starting to shift-left and integrate performance tests into sprints. The benefit of this approach is that performance engineers can identify potential problems before it’s too late and fixes can be deployed without delaying a release.

Protocol-based Load Tests

Protocol-based load tests simulate traffic at the API or protocol level by making requests and measuring responses. Until recently, protocol-based load tests were the only option for load testing web applications at scale. The most popular protocol-based load testing tool is JMeter, which is an open-source solution that has been around for years.

Suppose you want to ensure that a user dashboard loads within 15ms. Protocol-based tests would mock session variables to “sign in”, hit the `/dashboard` path, and measure how long it took to download the HTML, CSS and images from the server. Engineers would analyze server logs to see what resources took the longest to load and identify bottlenecks.

Protocol-based Pros

• They have fast run times since they’re making a simple request to a server.
• They have low resource usage since the cost of a simple request is very low.

Protocol-based Cons

• They do not execute JavaScript code and cannot accurately test JavaScript-heavy applications, such as single-page applications (SPAs).
• They require correlating the tests with lengthy server logs to identify any bottlenecks or performance issues — a time-consuming process.
• Test engineers must account for dynamic sessions and cookies to build many load tests, which translates to longer creation times.

Browser-based Load Tests

These spin up actual browser instances and instruct the browsers to go through real user workflows. While it’s possible to set these environments up in-house, falling cloud computing prices have made it more affordable to run in the cloud. LoadNinja provides one of the most advanced browser-based testing tools with scriptless record and replay.

LoadNinja Record and Replay Functionality – Source: LoadNinja

Suppose you want to test the same user dashboard as the protocol-based test. Browser-based tests would involve recording a user's actual behavior in the browser and replaying that behavior across thousands of real browser sessions running in the cloud. That way, you can see a more detailed picture of actual performance and find bottlenecks.

Browser-based Pros

• They provide a complete picture of the end-user experience by simulating an actual browser session rather than only requests.
• Developers can dive into each virtual user session to diagnose performance bottlenecks and other issues without server logs.

Browser-based Cons

• The cost of spinning up a virtual browser session can be relatively high and difficult to scale, especially if it’s done in-house.
• It takes more time to spin up a virtual browser session and run a test than hit a server endpoint without parallel testing capabilities.

Creating a Load Testing Plan

The best load testing plans incorporate both protocol-based and browser-based load tests to realize all of the pros and eliminate all of the cons.

Protocol-based tests are cost-effective for frequent runs. For instance, you can run JMeter tests with each minor commit to a repository as a sanity test to ensure you're not introducing any large bottlenecks. These tests are also great for load testing APIs endpoints that don't involve browsers, including endpoints for single-page applications (SPAs).

Browser-based tests are comprehensive options for less frequent runs. For example, you can run LoadNinja tests with each major commit to identify any bottlenecks. These tests ensure that critical user workflows, such as a sign-in or purchasing sequence, work properly before each production deploy. They're also great for ensuring that client-side apps are performant.

While conventional load tests were run just before a deployment, Agile development teams should consider incorporating performance tests earlier in the development cycle. These shift-left principles can help identify bottlenecks earlier and fix them without delaying a release or dealing with the repercussions of slow or error-prone software.

The Bottom Line

Load testing is critical to ensuring a positive user experience. Using both JMeter and LoadNinja, you can build a cost-effective test suite that's capable of covering all use cases. JMeter is best for rapid testing on a frequent basis or testing API endpoints, whereas LoadNinja is best for comprehensive testing of browser-level performance.

In addition to scriptless record-and-replay and extensive diagnostic tools, LoadNinja’s Jenkins and Jira plugins make it easy to integrate into any Agile development process. Anyone can record load test scripts with minimal programming knowledge and incorporate them into a CI/CD workflow to ensure that no bottlenecks reach production users.