How KoreroPlatforms (Powered by DiGiSPICE) built a multi-tenant SaaS architecture for their enterprise customers using AWS Organizations

DiGiSPICE is known to its customers as a thriving, innovation-led organization with extensive experience in running telecom services in India (Spice Telecom). They help enterprises in their digital transformation journey by harnessing the power of mobile and using technology to improve the connected customer experience.

With the above mission, DiGiSPICE has built a communications platform named KoreroPlatforms and offered this as a service for enterprises. Email is one of the supported communication channels as part of KoreroPlatforms offerings. Using this service, enterprises can send notifications to customers, such as transactional messages, alerts, reminders, promotional content, and newsletters. This also enables enterprises to help customers keep in touch with their activities via a managed campaign.

This post shows how KoreroPlatforms used an account-per-tenant model using AWS Organizations to leverage a more “siloed” Software as a Service (SaaS) model to support the unique tiering needs of its customers.

Prerequisites

This paper builds on the concepts and isolation models recommended in the SaaS Tenant Isolation Strategies whitepaper. A review of the different isolation strategies and the responsibilities that come with each model will help you make the right choice for your SaaS platform. This post is directed towards independent software vendors (ISVs) who deliver SaaS products, as well as solution architects who design systems that have an AWS account per tenant model. The architecture described is suitable for requirements that scale to thousands of tenants, but it’s unlikely to scale to hundreds of thousands or millions of tenants.

Solution overview

The original KoreroPlatforms architecture relied on a model where all of the platform’s components were deployed in a “pooled” SaaS model in which the entire tenant infrastructure was shared by the tenants running within a single AWS account. The following represents a list of the solution components that were part of the KoreroPlatforms architecture:

Portal Web Front End: This provides features and capabilities to Enterprise tenants so that they can onboard themselves onto the KoreroPlatforms in self-service mode. Enterprises can manage their email recipient lists, messages, and campaign templates, and can generate reports.

Application Backend: This provides backend functionality to Portal web application features.

CPaaS API Manager: This provides fully featured simple HTTP JSON-based APIs that work in any programming language. Application backend layer communicates with the CPaaS API Manager layer for interacting with AWS services via the AWS SDK for Java. For example, Amazon Simple Email Service (Amazon SES) for sending and receiving emails.

AWS Services used in this solution

Now, let’s look at how these components landed in AWS architecture. The diagram in Figure 1 provides an overview of the moving parts of this system, as well as how they were deployed on different AWS services.

Figure 1: KoreroPlatforms previous architecture

In this diagram, you’ll notice that KoreroPlatforms used Amazon EC2 Auto Scaling to have scalable compute capacity for their workload. The Portal Web Front End, Application Backend, and CPaaS API Manager components were all deployed on Amazon Elastic Compute Cloud (Amazon EC2) instances.

The notion of tenancy was supported across these different parts of the KoreroPlatforms model. Onboarding was initiated by the customer through the Portal Web Application. The configuration of the tenant was managed by the Application Backend services. The CPaaS API Manager was also deployed as a separate service running on Amazon EC2.

The database shown in this diagram represents Amazon Relational Database Service (Amazon RDS) that is used to store and configure tenant data. It holds all of the information required to acquire and process all of the tenant specific requests.

KoreroPlatforms uses Amazon SES to send email, including transactional, marketing, or mass email communications. Amazon SES sends event notifications to an Amazon Simple Notification Service (Amazon SNS) topic to enable the monitoring of emails that are sent using Amazon SES and to generate reports. Both the logging and monitoring of the Korero platform are conducted using Amazon CloudWatch.

Initially, KoreroPlatforms started building the email communication platform using the above-mentioned AWS services within a single AWS account. As more enterprises started onboarding, KoreroPlatforms received new requirements from some enterprises to be able to send emails directly using an SMTP client. Furthermore, KoreroPlatforms wanted to maintain each enterprise email sender reputation individually, which wasn’t possible in single account.

To solve these use cases, KoreroPlatforms decided to go with the Hybrid SaaS deployment model and use separate infrastructure resources for the Amazon SES service for each tenant. This let them offer distinct, per-tenant experiences that couldn’t be achieved in the pool model. To silo these resources and achieve the goals of their customers, KoreroPlatforms opted to silo the Amazon SES part of their experience using separate accounts for the Amazon SES portion of their system. This shifted them to a model where Organizations would be used to centralize the management of their siloed accounts.

How KoreroPlatforms made it work

Setup a new account with Organizations

As a first step, KoreroPlatforms created a new account and set up a new organization in this account using Organizations. Elements of an organization include a root (the recognized boundary of the organization), the management account, and member accounts organized into organizational units.

The diagram in Figure 2 provides a view of the new architecture. Here, the existing Korero components, such as Portal web front end, Application backend, and CPaaS API Manager, would still reside in the same pooled model that they used before (running in the same account). The new twist here is that we now have a separate account that is used to run the Amazon SES portion of the experience.

Figure 2: KoreroPlatforms architecture using Organizations

Here, you’ll see on the new approach that was used to support having separate accounts for the Amazon SES elements of the architecture. This is where the team relied on Organizations to support this new model, thereby creating a separate member account to support each tenant. Then, we have a management account that is used to aggregate events from these members accounts using Amazon SNS.

Using Amazon SNS in this management account allowed KoreroPlatforms to introduce a centralized monitoring mechanism where Amazon SES event notifications were published from each tenant accounts to a single Amazon SNS topic configured in the management account

This approach gave KoreroPlatforms the flexibility to configure the Amazon SES service and SMTP credentials per enterprise in an account dedicated to that tenant. It also provided the ability to maintain sender reputation individually for each tenant, and it still allows KoreroPlatforms to manage and operate these environments through a single pane of glass.

How KoreroPlatforms automates new tenant onboarding workflow with these changes

Automating the creation, configuration, and provisioning of tenant resources is fundamental to creating a rich, scalable multi-tenant experience. Follow this link for understanding best practices to onboard new tenant.

Let’s have a look at code changes in CPaaS API Manger layer to use Organizations in the KoreroPlatforms stack. Remember that the CPaaS API Manager is deployed to an account, which isn’t a member of the organization.

The following workflow is triggered when an enterprise tenant registers on the Web Application Portal and their onboarding process starts:

Figure 3: Enterprise onboarding Workflow process using Organizations

The process of account creation and configuration for each enterprise tenant on the KoreroPlatforms involves following steps executed from CPaaS API Manager Layer:

Step 1: Access Management Account: To create an account for enterprise, the first step is to access the organization’s management account programmatically from the CPaaS API manager layer. This is done using AWS Well-Architected guidance to use temporary credentials based on cross-account role assumption.

Step 2: Create and Verify New Member Account: After accessing the management Account, the CPaaS API manager layer performs the following steps:

2.1. Create a new member account using CreateAccount API.

When a member account is created using Organizations, it automatically creates an IAM role named OrganizationAccountAccessRole in the account. This role has full administrative permissions to that member account. This role is configured to grant that access to the organization’s management account

Please follow this link for more details on this concept.

2.2. Because CreateAccount operates asynchronously, this request completion status is checked by calling the DescribeCreateAccountStatus operation in a while loop before proceeding to the next steps.

2.3 Move the member account into an OU by calling MoveAccount API.

Step 3: Configure services in a Member Account: After the member account is created, verified, and moved to the required OU, the next step is to configure resources in the member account. To configure the newly created account, the CPaaS API manager layer uses the management account temporary credentials from Step 1 and assumes the role (OrganizationAccountAccessRole) created in the member account in Step 2. To assume a role, AssumeRole API from AWS Security Token Service (AWS STS) is used. Follow this link to understand the concept related to using the AWS Identity and Access Management (IAM) role for cross account access concepts.

After assuming the OrganizationAccountAccessRole role, build the new account’s networking, IAM, and Amazon SES service configurations for sending emails.

After moving the member account into an OU, OrganizationAccountAccessRole permissions are reduced to match the anticipated use of this role going forward.

The post, How to Use AWS Organizations to Automate End-to-End Account Creation, contains more detail about the above steps described in KoreroPlatforms process.

Step 4: Centralized email events monitoring in Management Account: KoreroPlatforms implemented centralized monitoring in the management account by publishing Amazon SES event notifications from all of the tenant accounts to an Amazon SNS topic configured in the management account. KoreroPlatforms achieves this by implementing Amazon SES configuration sets in the member account by calling API CreateConfigurationSet. Set the event destination to the Amazon SNS topic configured in the management account by calling API CreateConfigurationSetEventDestination

To allow Amazon SES from the member account publish to a topic that is owned by the management account, KoreroPlatforms follows these recommendations.

This lets KoreroPlatforms receive all of the member accounts Amazon SES events callback notifications for email sending metrics, including the number of sends, deliveries, opens, clicks, bounces, rejects, rendering failures, and complaints to the management account and take required actions centrally.

Step 5: Add and Verify Domain in the Member Account: Adds a domain to the list of identities for your Amazon SES account and attempts to verify it using VerifyDomainIdentity API call. After the domain is added and verified, KoreroPlatforms can create a DKIM ID per tenant account per each domain. Use VerifyDomainDkim API to return the set of DKIM tokens for a domain identity.

Once the above-mentioned steps are configured, the tenant account is ready to send the email. This enables KoreroPlatforms to partition and manage each enterprise’s emails separately and independently from each other. It also enables the ability manage service quotas as per each enterprise requirement. Additionally, by using AWS Resources Group, Amazon SES resource Tags, and Cost Allocation tags features, they plan to achieve increased visibility on Amazon SES service costs per domain in the member account level.

Summary

By implementing these organizational changes and using a multi-account strategy through Organizations, KoreroPlatforms achieved the following in their architecture:

• Defined a shared responsibility model of cost, security, compliance, and runtime in AWS Accounts
• Separate workload accounts for all of the enterprise tenants
• Separate sandboxes for the developers to work, experiment, and train in
• Separate accounts for PoC´s and Demos
• By having Amazon SES service configured per each member account, KoreroPlatforms can manage service quotas separately for each enterprise tenant
• KoreroPlatforms now has better control over email hygiene and can offer an SMTP account at each tenant level, which wasn’t possible before

On any given day, KoreroPlatforms is sending approximately 5 million emails on behalf of their enterprise customers, and they are increasing the volume rapidly.

Author: