Why Should You Load Test?
Developers and test engineers use many different types of tests. Unit tests ensure that a piece of code returns the right result, integration tests ensure that pieces of code work together and user interface tests ensure that the visual elements work across different devices and platforms.
Load tests, also known as performance tests, are often the most neglected type of test. If functional tests pass, it's easy to assume that the application is ready to launch. The problem is that, while some pieces of code may function, they may not scale well to hundreds or thousands of users.
For example, suppose that a developer writes a piece of code that contains an N+1 query that puts unnecessary load on a database. The query will return the correct result and may pass functional tests, but when the query is executed thousands of times, it could overload the database and crash the application.
The worst part is that these issues may not be noticeable until it's too late. For example, an e-commerce website could work just fine until Black Friday or a startup's web application might work perfectly until it gets picked up on Product Hunt. These are the wrong time for something to go wrong!
LoadNinja cuts down on load testing time by enabling test engineers to record and instantly reply load tests without tedious correlation or coding. Since we use real browsers, you also don't have to worry about adjusting scripts with dynamic elements like session IDs and cookies.
Performance Metrics to Watch
There are many different ways to measure the performance of an application. While response times are the most obvious metric, they don't tell the whole story when it comes to performance. For example, average response times may be low, but there could be a significant number of outliers with unacceptably high response times.
Before looking at load testing metrics, it's important to consider how to conduct load tests. Most load testing tools use protocol-based approaches that mimic requests and responses. It's important to test with real browsers in order to gain the most accurate insights — especially with front-end heavy apps.
Let's take a look at six of the most important metrics to watch and the value that they provide.
- Peak response time measures the roundtrip of a request/response cycle, but focuses on the longest cycle rather than taking an average. High peak response times help identify problematic anomalies.
- Error rates measure the percentage of problematic requests compared to total requests. It's not uncommon to have some errors with a high load, but obviously, error rates should be minimized to optimize the user experience.
- Concurrent users measure how many virtual users are active at a given point in time. While similar to requests per second (see below), the difference is that each concurrent user can generate a high number of requests.
- Throughput measures the amount of bandwidth, in kilobytes per second, consumed during the test. Low throughput could suggest the need to compress resources.
How to Analyze the Data
Most load testing tools provide a visualization that breaks down response times, error rates, and other key performance indicators. In addition, they may provide statistical data to support the charts that provide insights into averages, outliers, and other helpful metrics.
Example of LoadNinja Analytics
LoadNinja goes a step further by helping test engineers diagnose the actual bottlenecks. By breaking down performance by step and action, you don't have to dig deep to find the culprit for performance issues. Our real-time VU Inspector and VU Debugger make it very clear where the issues lie.
In addition to analyzing the data, it's important to ensure that it's relevant and up-to-date. Load testing should be integrated into continuous integration cycles to quickly identify issues before they reach production. For example, many development teams include it as part of a nightly build.
The Bottom Line
Load tests are just as important as functional tests in the Agile development process. By incorporating load tests into CI processes, you can avoid costly downtime during peak usage and ensure the best user experience at all times. LoadNinja makes the process much easier from start to finish.