Let's take a look at the most popular free and open source load testing tools – as well as premium tools that are worth the cost – to help you find the best option.
It's easy to overlook load tests when functional tests pass, but applications perform differently under load, and even slowdowns in production can be extremely costly.
Should You Use Open Source Tools to Load Test?
Open source load testing tools aren't always ideal, but they’re better than no load testing at all. A modest investment in one can pay long-term dividends. Regular load testing means more reliable performance and fewer slowdowns in production.
That said, there are a few limitations to these solutions:
- Virtual users originate on your servers, which means you're testing under ideal conditions compared to third-party load testing environments. In addition, you must build and maintain the underlying infrastructure.
- There’s no support if you have trouble implementing or maintaining the solution, which means you could end up spending engineers hours. There’s also no guarantee the project will continue to be supported over the long term by the community.
It’s important to keep these factors in mind when opting to use open source load testing tools. While they tend to have a lower upfront cost, they may not have the right feature sets for your use cases and may have a higher total cost of ownership.
JMeter is the most popular open source load testing tool with almost as many features as commercial tools. You can record test scripts, run large-scale load tests, and view various performance metrics in easy-to-understand charts. There’s also an active community that makes it easy to find everything from plugins to answers to common questions.
Apache JMeter Test Script Recorder – Source: Wikipedia
Taurus isn't so much a load testing platform as a tool for automating test scripts, from JMeter to Selenium. Using the easy-to-understand YAML language, you can quickly build tests and incorporate them into a continuous integration environment. You can use the same YAML language to generate reports and interface with third-party tools.
Here’s an example of a Taurus test script for running a scenario:
- concurrency: 100
You can run the scenario by executing the command `bzt quick_test.yml` in a terminal or in a continuous integration environment.
Locust is an open source load testing tool that lets you define user behaviors in Python and send millions of simultaneous users to your application. With its beautiful user interface, you can easily see key performance metrics, charts, failures, exceptions and other information. The only drawback is that it requires knowledge of the Python programming language.
Locust Load Testing Statistics - Source: Locust
Gatling scenarios are easy-to-understand for most developers:
var httpProtocol - http
var scn = scenario("BasicSimulation")
As with Taurus, these tests can be easily executed on the command line or integrated into continuous integration environments.
The Grinder is a Java-based load testing framework that uses Jython, a Java implementation of the Python programming language. If you're developing Java APIs, the load testing framework may be a good choice for developers already familiar with the language and concepts. Test engineers can also write Grinder scripts using the Closure language.
When to Use Commercials Tools
There are many different premium load testing tools that reduce time and cost of load tests. When evaluating these solutions, look for solutions within your budget that provide a robust set of features to help you accomplish your objectives — from the technical knowledge of test engineers to the frameworks you use.
LoadNinja is a leading premium load testing solution that makes it easy to record and replay tests across thousands of real browsers in the cloud. Test engineers and developers can access actionable browser-based metrics and diagnostics, including navigation timings, straight from real browsers and DOM-level access to each virtual user session.