Importance of Service Level Agreement for SaaS Providers

By Ashish Kathapurkar, Sr. Partner Development Manager, SaaS at AWS

One of the fundamental changes in adopting a software-as-a-service (SaaS) model is moving from a product delivery to a service-oriented mindset. SaaS provides a 24×7 digital channel between you and the end customers.

IDC’s SaaSPath 2020 survey polled 1,915 organizations across five continents to collect information on SaaS adoption and vendor preferences. It found that 1) data security, 2) value for price, and 3) ease of integrations as the top SaaS vendor attributes that buyers look at when evaluating a SaaS solution.

The same survey also found that guaranteed service level was the number one driver of increased investment in SaaS applications. One way to ensure data privacy and value for price is by way of SLAs. Traditionally, SLAs have been provided for service availability (uptime), case reporting, and resolution times.

As a Sr. Partner Development Manager at Amazon Web Services (AWS), I have worked with many organizations on their journey to build and operate SaaS solutions on AWS. I work with senior business leaders to provide business guidance in areas such as product planning and monetization, go-to-market and strategy, trials, SLAs, and customer success.

This post looks at how service level agreements (SLAs) aren’t just a measure of availability, but a tool that helps measure if the service is meeting the requirements of the customer-independent of the underlying architecture. Ensuring the application meets end-user expectations through SLAs is important to business growth (acquiring customers), reducing churn, and customer adoption of additional service features.

In a multi-tenant architecture, you have to think about application performance through the lens of a shared resources model where your experience (SLA) as a tenant could be impacted by other tenants. If we want the goodness of a multi-tenant model, you have to figure out what it means to express SLAs for this experience.

To develop SLAs for a multi-tenant environment, you need to create profiles or personas of the different tenants and determine how to draw lines—in this case based on SLAs—between the different types of tenants.

Ultimately, you should offer different experiences based on tenant profile and monetize them separately, aligning value with cost for your customers. This will let you address more market segments without undermining the revenue and margins of your business. Once you have this profile of tenants, you can think about how SLAs might be part of your overall tiering equation.

SLAs in a Multi-Tenant Architecture

In a single tenant architecture, as long as the system is designed for elasticity, reliability, and availability, resources are not shared across multiple tenants. Thus, it’s easy to meet end-user expectations of performance, latency, and transaction processing for all tenants.

In a multi-tenant architecture, resources are shared across multiple tenants. The more resources are shared, the more chances there are for one tenant to impact the experience of another. Any activity from one tenant that puts heavy load on the system, for example, has the potential to impact other tenants. Thus, a noisy neighbor can affect the service performance and the end-user experience for other tenants.

Multi-tenant architecture changes the landscape of service level agreements. When resources are shared by tenants that have unpredictable usage patterns, you now have to think about the collective experience of all your tenants—instead of a single customer.

Let’s examine this from an e-commerce application perspective. If you use catalogue size—defined by number of items—as a unit to assign tenants, then large and small catalogue size tenants can possibly share resources. In reality though, as illustrated in Figure 1, the traffic pattern and order processing can look very different for small, medium, and large catalogue tenants.

Figure 1 – Multi-tenant order processing profile.

In such as a scenario, the rate of order of processing can have several thousand dollars of impact and a diminished value provided by your service. Ensuring an order processing rate irrespective of the size of catalogue becomes an important value element of the end-user experience.

It is thus important to define the metric “rate of order processing” first in terms of end-user experience, measure it by various methods, and then make it the corner stone of your business model.

SLA Strategies

SLAs need to be woven into the fabric of the business strategy and architecture. Business and technical teams cannot work in silos in their understanding of end-user experience.

Sales, marketing, product, engineering, and customer success teams have to define SLAs in a value framework. Let’s look at four ways in which SLAs can be embedded into the service.

1. Service Tiering

Today, service tiering is used to align service price to usage, users, or features. It is not necessarily aligned to the end-user expectations.

Tiering, as shown in the table below, can be structured based on SLAs you are willing to provide to your tenants. As an example, Premium tier tenants who value order processing or faster inventory updates would be willing to pay a higher price. Price becomes an exchange rate that tenants are willing to pay for the value, and SLAs become a measure of the value provided by the service.

SLA-based tiering makes the business independent of the market you are addressing—Enterprise vs SMBs. Tenants are subscribing to a tier where they think they’ll get the most value that aligns to their business outcomes. SaaS architecture teams can choose the right architecture and underlying services that meets the SLAs.

2. SLA Framework

SaaS businesses spend a lot of time thinking through and creating SaaS frameworks to help improve their processes. Frameworks help in understanding, learning and growing the business.

The framework, as shown in Figure 2, is primarily used by product managers to build additional service features, measure feature adoption across customer segments, and iterate quickly to fix feature issues.

SLAs need to be part of the measurement of this framework. To quote Lord Kelvin, an eminent physicist best remembered for his talent for theoretical mathematics, “If you cannot measure it, you cannot improve it.”

Figure 2 – SLA framework.

It’s important to be able to measure SLAs over a period of time and make necessary adjustments to systems and processes in order to meet the SLAs. This helps identify patterns and track how you are performing against the stated goals.

In a multi-tenant SaaS model, SLAs need to be carefully crafted and selected. Once the set of SLAs are agreed upon, those SLAs per tenant should be collected, measured, and reported.

SLAs, per tenant, should be measured against the stated service objectives over a period of time. As an example: measure successful orders processed in the last 60 minutes divided by all valid orders in the last 60 minutes.

A per tenant dashboard should be created to track the service performance against the SLAs. Architecture teams should look at the metrics across their tenants and make tradeoffs according to the current and future needs of the business.

3. Business Impact

The more flexibility you are able to provide the business, the more likely it is you’ll be able to rapidly respond to the diverse requirements of tenants—each of which may have their own scale, performance, and consumption profiles.

As an example, for a usage-based offering, the ability to track usage per tenant is important to observe the overage and take necessary action. There could be material impact for not charging or incorrectly charging the tenant for the overage. Incorrect billing for the usage can lead to a higher churn rate causing further material impact.

Technical teams need to make architectural choices, such as use of containers, serverless, and databases, keeping in mind the SLAs (if tiering is the way the service is packaged).

Architecture teams should test for SLAs during load and stress testing to see the impact on user experience. This data can be used to devise policies that determine how you’ll handle tenants that may be exercising more load or exceeding their quota (if quotas are part of the service packaging).

This means instrumenting your code to observe tenant traffic patterns over time and using SLAs, such as order processing times, as a measure to right size the service. Keeping SLAs as a measure of success allows you to balance the cost and operational efficiency of SaaS with the reality that some tenants require their own instance of the SaaS environment to meet their SLAs.

4. SLA Reporting

Once you have the ability to measure SLAs for all tenants, you should plan to create reports that can be used during business review meetings.

There can be two areas with different reporting objectives:

• Tenant-centric – Helps you determine if you’re meeting stated SLA goals.
• Application-centric – In a multi-tenant environment, you can identify the noisy neighbor problem and make necessary system adjustments to measure application performance.

SLA reports need to be surfaced up to C-level management and made part of the organizational culture especially customer success team.

Customer success teams can send quarterly reports that reflect how the service performed and the value customers derived from it. This can be a great resource during service renewal time, reduce churn, and for service expansion.

Conclusion

Effective SaaS companies must be more proactive, act like a trusted advisor, and align their business outcomes to whom they serve.

Service level agreements (SLAs) should be used by SaaS providers to understand their role and how it contributes to customers’ business outcomes. Metrics such as uptime and reliability are table stakes for any SaaS provider.

SLAs need to align metrics to outcomes like customer satisfaction, contribution to revenue growth, churn rate, and lead to service expansion. If this new approach is not adopted, SaaS providers will continue to focus on metrics and generate reports that are of very little use to their customers.

About AWS SaaS Factory

AWS SaaS Factory helps organizations at any stage of the SaaS journey. Whether looking to build new products, migrate existing applications, or optimize SaaS solutions on AWS, we can help. Visit the AWS SaaS Factory Insights Hub to discover more technical and business content and best practices.

SaaS builders are encouraged to reach out to their account representative to inquire about engagement models and to work with the AWS SaaS Factory team.

Sign up to stay informed about the latest SaaS on AWS news, resources, and events.