How to Set the Right SLA for Your Application

What’s an SLA?

Service level agreements (SLA) are designed to ensure the vendor and customer are on the same page with performance expectations. For example, an ecommerce shop may want to ensure its website doesn't go down during holiday shopping periods (for obvious reasons).

SLAs are common in many industries where performance is vital. For instance, most web hosting companies provide SLAs for their customers guaranteeing 99.99% uptime — or less than an hour of downtime per year. Any excessive downtime is often remedied with an account credit.

SLAs include several components, such as:

• Description of services
• Conditions of availability
• Each party's responsibilities
• Measurement standards (metrics)
• Reporting processes
• Dispute resolution processes
• Indemnification clauses
• How to update the agreement
   

How Do SLAs Work?

Vendors typically provide a standard SLA to their customers as a starting point for negotiation. After review by legal counsel, they modify the agreement to suit the customer, depending on the price point and nature of the application (e.g. how critical the technical requirements are).

Most vendors provide public metrics through an online portal, where customers can ensure their SLA is always being met. In mission-critical services, customers sometimes invest in third-party tools to objectively capture SLA performance. They typically disclose this in negotiations, but not always.

If there's a breach of the SLA, the customer is entitled to the remedies provided in the agreement. These remedies range from account credits for excessive downtime, up to larger financial damages designed to cover any resulting business losses. Again, the nature of these remedies depends on the business.
 

How to Set the Right Level

SLAs are highly dependent upon the specific business. For instance, a cloud-hosting company would have a very different SLA than a SaaS application that targets the healthcare sector. The goal is to reassure the customer without opening the door to abuse by either party.

Some key points to keep in mind:

• Focus on behavior: Select those metrics that motivate the right behavior by both parties. For example, vendors shouldn't incur a penalty under an SLA if a client didn’t provide information in a timely manner.
   
• Realistic expectations: Select metrics the vendor can control. For example, each extra decimal after 99% adds a lot of extra expectations for uptime, and 99.9999% may not be viable for some companies.
   
• Easy measurement: Select metrics that can be measured with no ambiguity. In addition, SLAs should specify how these measurements take place to avoid any confusion and ensure that both parties are on the same page.
   
• Limit metrics: Choose a few metrics that accurately capture the goal rather than using a large number of metrics that nobody has the time or ability to measure. 
   
• Define the metrics: Metrics may seem straightforward on the surface, but they often require strict definitions. For example, automated replies aren’t appropriate ways to meet response time metrics for customer support tickets.

You should have your legal team intimately involved in the SLA creation. Since they’re legal agreements, they may be arbitrated at some point in court. At the same time, engineers should also be involved, as they have the best insights into what’s technically feasible as far as metrics and promises.
 

Verify by Load Testing

The best way for software vendors to ensure they meet SLA requirements is to performance test their applications on a regular basis.

Conventional load testing occurs at the end of each development cycle, which makes it challenging to fix any defects in time. In many organizations, load testing may not be a part of the deployment process at all, just something that’s run from time-to-time before major events (such as holidays, where usage may spike).

A better approach is to integrate load testing into your Agile development cycle early. Specifically, by encouraging collaboration between performance engineers and developers. By shifting these tests left, developers can spot potential performance issues before they make their final code commits and avoid defects reaching production.

LoadNinja’s Analytics Page

LoadNinja makes this possible by dramatically reducing the time it takes to create load tests. Using the browser-based tool, performance engineers can quickly build tests that incorporate dynamic data without writing any scripts. These tests can then be incorporated into CI/CD pipelines.

In addition to these features, the platform runs load tests across a network of real cloud-based browser instances to provide the most accurate picture of performance. These browsers execute JavaScript and CSS – unlike protocol-based tests – and provide developers with more in-depth analytics.

Worth Getting Right the First Time

Service level agreements, or SLAs, are a critical component of the enterprise software sales cycle. Customers want to know that the software subscriptions they purchase don't suffer from potentially going down at any time, especially crunch time. Software vendors should carefully design these policies to minimize risk and leverage load testing to minimize the risk of non-compliance.