Let's take a look at the most popular free and open source load testing tools – as well as premium tools that are worth the cost – to help you find the best option.
It's easy to overlook load tests when functional tests pass, but applications perform differently under load, and even slowdowns in production can be extremely costly.
Should You Use Open Source Tools to Load Test?
Open source load testing tools aren't always ideal, but they’re better than no load testing at all. A modest investment in one can pay long-term dividends. Regular load testing means more reliable performance and fewer slowdowns in production.
That said, there are a few limitations to these solutions:
- Virtual users originate on your servers, which means you're testing under ideal conditions compared to third-party load testing environments. In addition, you must build and maintain the underlying infrastructure.
- Most open source load testing tools are protocol-based rather than browser-based, which means that there's no JavaScript execution or HTML/CSS rendering taking place. Browser-based tools usually aren't cost effective for in-house tests.
- There’s no support if you have trouble implementing or maintaining the solution, which means you could end up spending engineers’ hours. There’s also no guarantee the project will continue to be supported over the long term by the community.
It’s important to keep these factors in mind when opting to use open source load testing tools. While they tend to have a lower upfront cost, they may not have the right feature sets for your use cases and may have a higher total cost of ownership thanks to regular maintenance, time spent, and amount of resources required. This can quickly add up and what was once “free” is quickly not “free” anymore.
JMeter is the most popular open source load testing tool with almost as many features as commercial tools. You can record test scripts, run large-scale load tests, and view various performance metrics in easy-to-understand charts. There’s also an active community that makes it easy to find everything from plugins to answers to common questions.
Apache JMeter Test Script Recorder – Source: Wikipedia
Taurus isn't so much a load testing platform as a tool for automating test scripts, from JMeter to Selenium. Using the easy-to-understand YAML language, you can quickly build tests and incorporate them into a continuous integration environment. You can use the same YAML language to generate reports and interface with third-party tools.
Here’s an example of a Taurus test script for running a scenario:
execution:
- concurrency: 100
ramp-up: 1m
hold-for: 5m
scenario: quick-test
scenarios:
quick-test:
request:
- http://demosite.com
You can run the scenario by executing the command bzt quick_test.yml in a terminal or in a continuous integration environment.
Locust is an open source load testing tool that lets you define user behaviors in Python and send millions of simultaneous users to your application. With its beautiful user interface, you can easily see key performance metrics, charts, failures, exceptions and other information. The only drawback is that it requires knowledge of the Python programming language.
Locust Load Testing Statistics - Source: Locust
Gatling is a developer-focused load testing tool for web applications. Using its JavaScript API, developers can quickly write complex load tests and integrate them into continuous integration and deployment processes. Companies looking for a web-based interface with real-time monitoring can also check out its commercial FrontLine enterprise product.
Gatling scenarios are easy-to-understand for most developers:
var httpProtocol - http
.baseUrl("")
.acceptHeader()
.doNotTrackHeader()
.acceptLanguageHeader()
.acceptEncodingHeader()
.userAgentHeader()
var scn = scenario("BasicSimulation")
.exec(http("request_1")
.get("/")
As with Taurus, these tests can be easily executed on the command line or integrated into continuous integration environments.
The Grinder is a Java-based load testing framework that uses Jython, a Java implementation of the Python programming language. If you're developing Java APIs, the load testing framework may be a good choice for developers already familiar with the language and concepts. Test engineers can also write Grinder scripts using the Closure language.
How to Choose the Best Load Testing Approach for Your Organization
When to Use Commercials Tools
There are many different premium load testing tools that reduce time and cost of load tests. When evaluating these solutions, look for solutions within your budget that provide a robust set of features to help you accomplish your objectives — from the technical knowledge of test engineers to the frameworks you use.
Keep in mind when selecting these commercial tools their inherent benefits that may not be as upfront as the benefits of open source solutions. These tools often will make creating and automating tests easier with more documentation and support to guide you through the process. However, it’s important to be aware which tools are protocol-based or browser-based, as they can be significantly different in terms of automation, time spent designing tests, and the learning curve to learn the application.
LoadNinja is a leading premium load testing solution that makes it easy to record and replay tests across thousands of real browsers in the cloud. Test engineers and developers can access actionable browser-based metrics and diagnostics, including navigation timings, straight from real browsers and DOM-level access to each virtual user session.