What's Continuous Integration/Continuous Delivery (CI/CD)? 

CI/CD is a process of continuously integrating and deploying new code into an existing codebase. They are usually said in conjunction but we’ve actually found that it’s pretty often that only one is happening, and not the other.

What is Continuous Integration?

Continuous Integration refers to the practice of integrating code into the main branch of your repository. By doing so, you can generally avoid the master code merge that amplifies the chance of this integration causing bugs and making debugging nearly impossible. At the point of integration, code is packaged, built, and tested via an automated process – but code is not released or deployed.

What is Continuous Delivery?

Continuous Delivery is the process of actually deploying these series of small integrations to users in an automated way. Continuous integration is an important concept and the foundation of CI/CD, but if code is not deployed to production automatically after the integration process has successfully passed, then we are not continuously deploying.

Why are Teams Looking to Automate?

Automation can help teams move faster. 

First we can start with faster development cycles. For agile and devops teams, this is key – automating tests can help teams deploy often, and deploy still with confidence.

Increase Test Coverage

Second is it allows us to truly increase our test coverage. At the end of the day, their will always be a limited amount of time to test a product, and teams with mainly manual testing will just not be able have sufficient test coverage across their application. Automation helps us achieve an increase in test coverage, without an increase in time. Build testing into the entire application lifecycle can increase quality without increasing test time when it comes time to deploy.

Faster Mean Time to Resolution

Lastly, we’re getting instant feedback in our development cycles. Automated testing can amplify instant feedback across multiple layers, so teams can get test results back to the development team for faster bug fixes and reduced time triaging.

Why Load Test in a CICD Environment?

Load testing is essential in your software development lifecycle. In terms of continuous integration and continuous delivery, integrating load testing processes can drastically shift the bottleneck away from performance testing, which for most teams, is the case.

This makes sense when you think about it because typically, getting up and running with traditional load testing tools takes a lot of time and effort to get right.  Scripting tests, especially performance tests, can be a very hard ordeal. Many business, QA and product teams expect almost every possible transaction to get scripted as realistically as possible and load test them across different scenarios by tweaking the right parameters. Many a time, this would also require the tester to data-drive their load tests, wherein external data is gradually incorporated into different user transactions to model a real-world scenario. 

Depending on the complexity of the application and the user transactions, the above could require anywhere between a few days to several weeks to orchestrate successfully. The end result is not only extremely time consuming and resource intensive for the team, but can also put your development and delivery on hold in the fast paced world of agile and devops workflows.

However, if you’re running CICD processes without including your load tests in that process, you’re likely missing a significant element in your evaluation of quality. To truly understand how your application is going to behave in production, it's essential to gauge how your application responds under load. Or minimally, to get a good idea of your overall application performance outside of your functional testing processes. 

By integrating and automating this process, you can ensure that you're retaining or increasing load testing coverage while reducing the time it takes to complete the performance testing process. Amplifying the feedback you can get from these load tests helps ensure that every change you make to your codebase is tested and fixed as early as possible. 

How to Automate Load Testing Processes

If you’re using tools like Jenkins, TeamCity and Bamboo, you're likely continuously integrating new or changed code into a shared repository and verifying these changes with an automated build. 

1. Make sure you're load testing tool supports these kinds of integrations
2. Understand which tests need to run & when - make sure that these test scripts are already created and ready to execute
3. Evaluate how frequently you want the tests to run

Begin identifying how the load testing tools you rely on can integrate with your CICD pipeline. This may be through native integrations or API-based, but either way, you should be able to set these up so that you can kick-off your load tests automatically. 

Automating Load Tests with LoadNinja

Automating tests in LoadNinja is easier than you'd think. Once you have the tests you'd like to include in your CICD processes recorded, you can integrate with a CI tool like Jenkins using the LoadNinja API.

Firstly, you'll need to grab your API Key. Then you can add the HTTP request plugin to your Jenkins instance. After these steps have been completed, you can check out the LoadNinja API documentation in SwaggerHub to identify how you want to use the API. You can use Swagger Inspector to test the API as well as get the response data you need to put into your Jenkins build (i.e. project name, project ID, account ID, etc.). Once you've filled in the required fields in the API call, you'll be able to send this call through the HTTP request plugin in Jenkins. Create a new build in Jenkins, paste in the URL, choose the HTTP mode, set the authentication, & add authorization in the header, where you'll add in the API Key. Then add the info in from the API Structure (gathered in Swagger Inspector) & make sure the content type is Application JSON. Once this configuration is set up, this is all you'll need to set up a full scenario execution within LoadNinja.