5 Open Source Load Testing Tools (and When to Go Commercial)

Developers and QA engineers incorporate lots of tests into their continuous integration and deployment workflow. While unit tests, integration tests and UI tests are commonplace, load tests are sometimes overlooked by teams that believe functional tests are enough. The problem is that applications perform differently under real-life conditions.

Even slowdowns in production can be extremely costly for businesses. In fact, a one-second delay in load time translates to a 7% reduction in conversion rates! Big events like Black Friday and Cyber Monday can also create large loads at a critical time for generating revenue, making load testing even more necessary to ensure a good user experience and maximize revenue for the business.

Let's take a look at the most popular free and open source load testing tools – as well as premium tools that are worth the cost – to help you find the best option.

It's easy to overlook load tests when functional tests pass, but applications perform differently under load, and even slowdowns in production can be extremely costly.

Should You Use Open Source Tools to Load Test?

Open source load testing tools aren't always ideal, but they’re better than no load testing at all. A modest investment in one can pay long-term dividends. Regular load testing means more reliable performance and fewer slowdowns in production.

That said, there are a few limitations to these solutions:

  • Virtual users originate on your servers, which means you're testing under ideal conditions compared to third-party load testing environments. In addition, you must build and maintain the underlying infrastructure.
     
  • Most open source load testing tools are protocol-based rather than browser-based, which means that there's no JavaScript execution or HTML/CSS rendering taking place. Browser-based tools usually aren't cost effective for in-house tests.
     
  • There’s no support if you have trouble implementing or maintaining the solution, which means you could end up spending engineers hours. There’s also no guarantee the project will continue to be supported over the long term by the community.

It’s important to keep these factors in mind when opting to use open source load testing tools. While they tend to have a lower upfront cost, they may not have the right feature sets for your use cases and may have a higher total cost of ownership.

JMeter

JMeter is the most popular open source load testing tool with almost as many features as commercial tools. You can record test scripts, run large-scale load tests, and view various performance metrics in easy-to-understand charts. There’s also an active community that makes it easy to find everything from plugins to answers to common questions.

Apache JMeter Test Script Recorder – Source: Wikipedia

Taurus

Taurus isn't so much a load testing platform as a tool for automating test scripts, from JMeter to Selenium. Using the easy-to-understand YAML language, you can quickly build tests and incorporate them into a continuous integration environment. You can use the same YAML language to generate reports and interface with third-party tools.

Here’s an example of a Taurus test script for running a scenario:

```

execution:

- concurrency: 100

            ramp-up: 1m

            hold-for: 5m

            scenario: quick-test     

scenarios:

            quick-test:

                        request:

                        - http://demosite.com

```

You can run the scenario by executing the command `bzt quick_test.yml` in a terminal or in a continuous integration environment.

Locust

Locust is an open source load testing tool that lets you define user behaviors in Python and send millions of simultaneous users to your application. With its beautiful user interface, you can easily see key performance metrics, charts, failures, exceptions and other information. The only drawback is that it requires knowledge of the Python programming language.

Locust Load Testing Statistics - Source: Locust

Gatling

Gatling is a developer-focused load testing tool for web applications. Using its JavaScript API, developers can quickly write complex load tests and integrate them into continuous integration and deployment processes. Companies looking for a web-based interface with real-time monitoring can also check out its commercial FrontLine enterprise product.

Gatling scenarios are easy-to-understand for most developers:

```

var httpProtocol - http

            .baseUrl("")

            .acceptHeader()

            .doNotTrackHeader()

            .acceptLanguageHeader()

            .acceptEncodingHeader()

            .userAgentHeader() 

var scn = scenario("BasicSimulation")

            .exec(http("request_1")

            .get("/")

```

As with Taurus, these tests can be easily executed on the command line or integrated into continuous integration environments.

The Grinder

The Grinder is a Java-based load testing framework that uses Jython, a Java implementation of the Python programming language. If you're developing Java APIs, the load testing framework may be a good choice for developers already familiar with the language and concepts. Test engineers can also write Grinder scripts using the Closure language.

When to Use Commercials Tools

There are many different premium load testing tools that reduce time and cost of load tests. When evaluating these solutions, look for solutions within your budget that provide a robust set of features to help you accomplish your objectives — from the technical knowledge of test engineers to the frameworks you use.

LoadNinja is a leading premium load testing solution that makes it easy to record and replay tests across thousands of real browsers in the cloud. Test engineers and developers can access actionable browser-based metrics and diagnostics, including navigation timings, straight from real browsers and DOM-level access to each virtual user session.