Modern SaaS architecture: A practical guide for software companies

The SaaS deployment model is one of the most critical architectural decisions in cloud computing because it determines the business model’s viability and scalability. At the least, your decision should take into account pricing, compliance, and operational constraints. If it doesn’t, it means you’re building a road without knowing the surrounding landscape.

• Single-tenant architecture: Each customer has dedicated compute, data storage, and cloud infrastructure. This model helps prevent noisy neighbor risk and provides data residency within specific regions.
• Multi-tenant architecture: Multiple customers share the same pool of logically isolated resources. You rely on mechanisms such as load balancers, autoscaling, tenant-based throttling, and quotas to avoid performance bottlenecks
• Hybrid architecture: This model combines single-tenant and multi-tenant models to deliver cost and operational efficiency, while providing robust protection for sensitive data.

Many SaaS teams opt for a multi-tenant SaaS architecture because it’s cost-efficient. While the choice makes business sense, teams should also make room for expansion by defining isolation boundaries. For example, maintaining proper control and data-plane separation allows you to deploy business logic separately from workloads. This way, you can expand more into different regions and be better prepared for enterprise opportunities that need dedicated infrastructure.

Another common mistake SaaS vendors make is thinking they can easily pivot between architectural models. The reality is that once you’re deploying SaaS architecture in a particular way, shifting your workloads and data to a different method needs substantial architectural rework. You’ll need to re-engineer your architecture, compute services, and data pipelines to meet new constraints.

• How do we handle noisy neighbor effects in a multi-tenant model if a high-volume customer spikes in activity?
• If one tenant’s environment fails, what impact is felt by other customers?
• What is the issue resolution time for a dedicated instance versus a multi-tenant environment?

With AWS, you can choose a deployment model to suit your requirements for operational costs, governance, and data security. You can use AWS Control Tower to set up multi-account environments, ready to deploy your SaaS model. Meanwhile, AWS Organizations lets you introduce account-level policies to govern SaaS applications as you scale, without rearchitecting account structures.

Observability is one of the SaaS architecture fundamentals—it shows how your software is performing. Observability shouldn’t be an afterthought when building SaaS platforms. The type of metrics and monitoring you will need in the future should be an up-front consideration.

For example, some teams only monitor SaaS instances for service uptime. However, this limits their ability to troubleshoot issues as they lack any significant actionable data. Your team won’t know where to start when services fail. Instead, you need inputs from observability signals in logs, traces, and metrics.

• Logs capture time-stamped events across multiple tenant-level touchpoints in your SaaS architecture. They tell you what happened when a situation arises.
• Traces follow a request’s path through the software in your SaaS model, showing the components it interacted with, timings, and any failures in the path. They surface patterns that answer where and help determine why an incident happened.
• Metrics provide an overview of specific measurable qualities of the system at a point in time or during a period. For example, they can show how much compute each instance is consuming and the response time of services.

Observability is the practice of consolidating logs, traces, and metrics to understand SaaS health with greater precision. Together, these tools form the intelligence your team relies on to quickly respond to service outages. Customers expect SaaS providers to uphold their service level agreements (SLAs), which have a lower tolerance for downtime. For example, if your SLA promises 99.99% uptime, your SaaS can only afford 4.38 minutes of total downtime per month. Anything more than that is considered a breach of agreement.

• Can we identify which specific tenant is consuming the highest percentage of database IOPS right now?
• If a customer reports a bug, can our support staff pull up a trace of only that customer's last 10 failed requests?
• When we deploy a horizontal scale event, can we see which tenants had traffic routed to the new nodes and if their tail latency improved?

Architecting observability into SaaS starts with knowing what you want to capture and instrumenting your code before selecting your tools. You can use Amazon CloudWatch to assign metrics to specific tenants. This allows you to gain complete visibility and address performance bottlenecks affecting specific SaaS customers. Meanwhile, Amazon CloudWatch Application Signals lets you implement distributed tracing across microservices and can map services deployed across regions. Software teams can also use Amazon OpenSearch Service to aggregate logs and derive actionable insights. OpenSearch Service includes machine learning to automatically detect anomalous patterns in logs, which can help accelerate remediation. 

SaaS teams might assume they’ve adequately isolated their tenants due to tenant ID checks. However, you also need to include database-level controls. That way, if your dev teams mistakenly feed data retrieval with the wrong ID or forget to include the WHERE clause, there are no accidental leaks or compliance breaches.

Before we dive deeper, let’s define what isolation is. At the conceptual level, isolation is the act of separating resources and infrastructure in the SaaS application architecture. However, there are distinctions between how the boundaries are implemented. We call this hard isolation and soft isolation.

• Hard isolation has dedicated infrastructure per tenant, with virtual private cloud (VPC) isolation, and account-level boundaries.
• Soft isolation has shared infrastructure between tenants, with logical boundaries through application-level checks or row-level database filters, and additional application-level IAM policies.

• Could the observability stack inadvertently expose personally identifiable information (PII) from one tenant’s data and display it in a dashboard visible to developers troubleshooting another tenant?
• Could a shared caching layer return an object belonging to Tenant A to a user from Tenant B?
• How does a spike in a tenant affect other tenants sharing the same cluster of resources?

AWS secures the underlying infrastructure for running workloads, offering various models of workload isolation, up to dedicated hardware instances. SaaS vendors manage the isolation of their critical workloads, sensitive data, and tenant access.

AWS offers managed services that support tenant isolation:

• You can use AWS Identity and Access Management (IAM) to set, enforce, and manage tenant-level access controls for role-based access.
• When designing databases, you can use Amazon RDS to assign schemas for each tenant. This provides considerably stronger boundaries than row-level tenancy. To gain greater isolation, choose per-tenant databases, and for even more isolation, use dedicated instances.
• Amazon Elastic Kubernetes Service (EKS) allows you to create namespaces and assign them to specific tenants. It complements hard isolation techniques such as node group isolation and network policies. 

SaaS companies are expected to comply with legally binding regulations, and some industries and jurisdictions are more complex than others. You might need to demonstrate compliance with data privacy, data security, and governance laws, such as the GDPR and HIPAA. Or, you might need to certify against security standards such as SOC 2 or ISO 27001 to meet customer needs.

To pass an audit, you must be able to present comprehensive logs, reports, and records in accordance with regulatory requirements. Manually generating audit evidence is time-consuming. SaaS providers often only realize the scale of such efforts six months after launch. By then, they’ve lost the opportunity to capture access records, security logs, and other statements that serve as audit evidence. When they’re required to submit audit records, they’ll need to somehow produce data that might not have been retained.

When building SaaS products, software companies that embed compliance into their architecture hold an undeniable advantage. They’re more likely to meet audit requirements and present favorably in enterprise contracts. In considering compliance, they addressed critical questions before commencing development.

• If a developer left the company today, could we prove their access to the production database was revoked within our defined SLA?
• In the event of a breach, how quickly could we produce a list of every user who accessed a specific customer's data in the last 90 days?
• Do we have a process to automatically flag when one of our third-party vendors loses their SOC 2 or ISO 27001 status?

Designing a scalable multi-tenancy SaaS architecture comes down to where tenant boundaries are enforced: either primarily in application code or consistently across identity, infrastructure, compute, and operations. Your identity decisions constrain your data strategy. Your data strategy influences your compute architecture, and your compute choices directly shape your daily operations and observability.

Architectures that embed tenant context consistently across these layers are generally easier to scale as customer requirements evolve. Strong architectural foundations reduce future rework while making it easier to support larger enterprise customers over time. To evaluate how your architecture aligns with these patterns, use the AWS Well-Architected SaaS Lens as an objective review mechanism before pursuing larger enterprise procurement opportunities.A Well-Architected SaaS Lens review can help you identify where your blocks stand today so you can build with confidence, reach customers faster, and grow your business through AWS Marketplace.

Explore the AWS ISV Accelerate program to put that readiness to work as you scale into larger enterprise markets.