What Are Assertions?

Most developers and test engineers are familiar with assertions in unit and integration tests. Each assertion is an independent, complete, and testable statement of requirements in a specification. They provide a foundation for building test cases that uncover inconsistencies, ambiguities, gaps, and non-testable statements in the specification.

In load tests, assertions verify that a server returns an expected result. For example, a “duration assertion” tests if a response was received within a specified time. Responses that take longer than the threshold "fail" since they don't meet performance specifications.

To cover your performance requirements, there are many different types:

• Response Assertions ensure that a server response contains an expected response code, header or other field value.
   
• Duration Assertions ensure that a server response was received within a given amount of time.
   
• Size Assertions ensure that a server response contains the expected number of bytes (e.g. less than a certain amount).
   
• XML/JSON Assertions ensure that a server response contains a valid XML/JSON document.
   
• HTML Assertions ensure that a server response contains valid HTML syntax.

Different contexts require the different specifications. For example, you may want the application to respond within a certain time under peak expected load, or ensure that peak load conditions don't result in malformed JSON or XML documents.

How They Fit into Load Tests

Many unit and integration tests are code-based. However, most load testing tools are GUI-based scripting tools (with the exception of Gatling).

For instance, JMeter is arguably the most popular load testing framework. The Java application requires a combination of GUI configuration and scripting to execute the load tests against a web application.

JMeter Response Assertion Dialog – Source: JMeter Documentation

There are four steps to using an assertion in the load testing process:

1. Add the Assertion – Assert that the server's response code will be a certain value.
2. Add the Pattern to Test – Add the pattern to match the server's response code to.
3. Add the Assertion Result – Assert that the server's response code should be '200 - OK'.
4. Run the Test – Run the test and see any failures.

These tests can be run in two ways:

• Protocol-based tests, such as those run by JMeter. They simulate load by sending GET/POST/PUT/DELETE requests through to a web server and measuring its responses.
   
• Browser-based tests, such as those run by LoadNinja. They generate traffic from real browsers running in the cloud, providing a more accurate picture of true performance.

Do browser-based tests at scale make sense?

Browser-based tests have been historically difficult to scale, because spinning up a browser is expensive. While they’re helpful for troubleshooting, it wasn't practical to run them at scale with each production deploy – until LoadNinja. Our browser-based testing platform makes it easy to spin up tens of thousands of browser instances at an affordable price.

Finding Assertions that Meet Your Expectations

The biggest challenge with assertions is coming up with the right performance expectations. For example, what's the right duration assertion to use? Or, how many virtual users do you need to generate? Consider these questions before writing load test assertions.

So for example:

• Google researchers recommend page load times of less than 100 milliseconds. Your brain processes visual memory at that speed, so it gives the illusion of instantaneous website response. While it may not be possible to load the entire page in that time, it's a good idea to ensure that something shows up (time to first byte).
• Load tests should focus on handling peak anticipated loads rather than finding maximums. Start by analyzing existing traffic patterns and come up with a realistic peak load scenario. Using that data, you can ensure your application is responsive under a realistic load rather than spending time and money optimizing for an unreasonably high one.

With browser-based testing, tests like these come back with accurate results. With JavaScript heavy web applications, protocol-based tests may seemingly perform well because the JavaScript code is never executed by a browser. But browser-based tests, like with LoadNinja, could uncover hidden bottlenecks in the code that provide insights into the true user experience.

Assertions in LoadNinja, aka Browser-Based Testing

LoadNinja assertions are known as validations designed to ensure that the application works as expected.

The first step in using validations is recording a script using the Script Recorder. Within the LoadNinja interface, you can use your web application as a user would, and each load test step is automatically recorded.

You can add validations in a few easy steps:

1. Select Add > Validation from the toolbar.
2. Select the Text, JavaScript or SLA validation type.
3. Configure the appropriate settings.
4. Click Test to ensure it’s configured correctly.
5. Click Add Validation to add the validation.

Adding a Validation in LoadNinja – Source: LoadNinja Documentation

There are three different types of validations:

• Text Validations ensure that web pages contain specified tests.
• Script Validations perform complex checks, such as confirming the presence of an element or an attribute’s class.
• SLA Validations verify that the service responds within a specified time frame.

If a validation fails, LoadNinja will stop running the script on a given virtual user and log the error. If the scenario’s On Error setting is set to Enter Debug Mode, the debug mode will be available for the failed virtual user.

You can learn more about using validations in the LoadNinja documentation.

Or, to learn about it firsthand, sign up for a free trial to test it out on your own application in real time.

The Bottom Line

Load tests are important for modern web applications – and much easier than they used to be. Using LoadNinja and assertions with browser-based testing, you can uncover errors that you never would have seen before. Easily record load tests, create assertions, and run the tests in real browser sessions. You can even integrate these tests with continuous integration and deployment servers alongside the rest of your test suite.