Account and VPC Considerations for VMware Cloud on AWS

By Schneider Larbi, Sr. Partner Solutions Architect at AWS

VMware Cloud on AWS is a service that is jointly engineered by VMware and Amazon Web Services (AWS) to allow customers to run VMware workloads on the global AWS infrastructure.

During the deployment phase of the VMware Cloud on AWS service, the Software Defined Data Center (SDDC) is connected to an AWS (or customer) account for seamless access to native AWS services.

In this post, I will provide guidance on which AWS account and respective virtual private cloud (VPC) to connect VMware Cloud on AWS to take advantage of native AWS service integrations.

Account Structure

There are two accounts required when setting up a VMware Cloud on AWS. The first is the VMware Cloud SDDC account. This is an AWS account that runs the SDDC or VMware Cloud on AWS resources. It is owned and operated by VMware.

The second account required is the AWS customer-owned account. It is owned, operated, and paid for directly by the customer if the customer chooses to utilize any AWS services within that account.

To successfully attach the AWS customer-owned account to the SDDC, the AWS account should have at least one VPC within that account. This attachment allows customers to use native AWS services to compliment whatever service they run on VMware Cloud on AWS.

Account Architecture

The diagram below depicts the architecture of how the two required accounts for setting up VMware Cloud on AWS interact.

Figure 1 – AWS account and VPC connectivity to SDDC.

As you can see, the Amazon Virtual Private Cloud (VPC) to the left is hosted within the AWS account that is managed and operated by VMware. Customers will have no access to this VPC.

The VPC to the right is hosted within the customer AWS account, and this account is what’s managed directly by the customer. Depending on the resources running within the VPC of that account, customers may have to pay for those services.

For example, looking at the architecture above, the customer will be responsible for paying for the Amazon Elastic Compute Cloud (Amazon EC2) instances within the connected VPC in the customer account.

As described in Figure 1, the Elastic Network Interface (ENI) provides a high bandwidth and low latency access to services within the connected VPC.

This allows virtual machines in the SDDC cluster to leverage native AWS services such as Amazon EC2, backing up virtual machines to Amazon Simple Storage Service (Amazon S3), and offloading database management to Amazon Relational Database Service (Amazon RDS) without traversing the public internet. The traffic traverses through the private AWS network backbone.

AWS Account Selection Considerations

There are many customers who run AWS environments that comprises several VPCs; some also deploy VMware Cloud on AWS to run other workloads. In these instances, customers ask us about the best VPC to use for connection to VMware Cloud on AWS.

The right VPC to use as a connected VPC depends on several factors. To begin, I will discuss some account deployment patterns for AWS Accounts.

Account Management: AWS Organizations

AWS accounts that host the connected VPC can belong to an AWS Organization, which allow customers to centrally govern their AWS Cloud resources.

If you are using AWS Organizations, it’s recommend you plan ahead which accounts you want to associate with VMware Cloud on AWS. You can then create an AWS Organization Unit (OU) and associate those with the accounts you reserve for VMware Cloud on AWS.

If you associate any Security Control Policies (SCPs) to the OU, ensure the SCP does not prevent or restrict the use of AWS CloudFormation. VMware Cloud on AWS uses a CloudFormation template to provision necessary resources in the connected account, specifically within the VPC for seamless access to native AWS services with the VMware SDDC.

Multi-Account Governance: AWS Control Tower and Landing Zones

There are also customers who utilize AWS Control Tower to manage the AWS Organization to enforce governance across multi-account environments. In such a scenario, we recommend you create a separate OU for accounts you want to use with VMware Cloud on AWS. On the OU, you need to ensure the guardrail rules that are used do not restrict CloudFormation from creating the necessary resources in the connected VPC.

The CloudFormation template used by VMware has the prefix vmware-sddc-formation, and the CloudFormation stack does the following in the connected VPC within the connected account:

1. It creates immutable identity and access management (IAM) roles in the VPC; namely RemoteRole, RemoteRolePlayer, RemoveRoleService, and BasicLambdaRole.
2. It creates an IAM policy called AmazonVPCCrossAccoutNetworkOperations for the above roles.
3. It creates AWS Lambda functions for event notifications.

Therefore, whether using AWS Organizations or a Control Tower with landing zones, you need to configure your policies to allow the account that will be associated with VMware Cloud on AWS to complete these operations in the VPC.

Standalone AWS Accounts

Standalone accounts can be added to an AWS Organization or Control Tower landing zone at a later time. If there’s a requirement to do so, ensure you have set your governance policies to allow you to run any native AWS service you may require in your connected VPC.

Also, guardrail policies or SCPs must allow AWS CloudFormation to work within the account.

Standalone accounts can also be used without being part of an AWS Organization or managed by AWS Control Tower with landing zones.

Connected VPC Considerations

As described in Figure 1, the connected VPC provides a high bandwidth and low latency access to the VPC, which allows customers to consume native AWS services.

On the available instance types used in VMware Cloud on AWS, VMware supports up to 25 GBps aggregate bandwidth.

If you have requirements to integrate native AWS services such as Amazon RDS or Amazon Redshift, and you require low latency access to these services, it’s recommended you deploy these services into the connected VPCs to allow your VMware workloads to utilize these services.

Additionally, some existing customers architect their environments with a shared services VPC or service provider VPCs. In this architecture, customers use one VPC to expose and share services and applications to other VPCs that we’ll term as “consumer” VPCs within a particular AWS Region.

This architectural model is powered by AWS PrivateLink, which creates a Network Load Balancer for applications within a VPC and an interface endpoint to the supported for the applicable AWS service.

This creates an ENI called the interface endpoint in the subnets within the VPC with a private IP address that serves as an entry point for traffic destined to the service.

You can configure your own services behind a Network Load Balancer in a service provider VPC, and other VPCs can consume that service as shown in the diagram below.

Figure 2 – AWS PrivateLink connectivity to service provider account.

Figure 2 describes how consumer VPCs from different accounts can consume VPC resources in a service provider account. It uses AWS PrivateLink to communicate with the target VPC.

AWS PrivateLink can work over a VPC Peering, a virtual private network (VPN), and AWS Transit Gateway connections to send traffic to a provider VPC in cases where consumer accounts and VPC reside in a different region.

If a customer is already using this architecture, they can simply integrate this architecture to VMware Cloud on AWS. If the customer is going to use a single SDDC or VMware Cloud on AWS environment, they can use the shared VPC or the service provider VPC as the connected VPC to VMware Cloud on AWS.

Figure 3 – AWS PrivateLink integration with connected AWS account/VPC with SDDC.

In Figure 3, a service provider VPC is being used as the connected VPC for VMware Cloud on AWS. With this model, VMware workloads running within VMware Cloud on AWS can still use native AWS services from the service provider VPC through the ENI between the VPC and VMware Cloud on AWS.

It’s worth noting the accessing native services from a connected VPC that’s in the same AWS Availability Zone as your SDDC will incur no intra-Availability Zone traffic charges.

Additionally, other VPCs can also use AWS interface endpoints in those VPCs to consume services within the connected VPC which serves as a service provider VPC. With the configuration of interface endpoints, you can access supported services from the connected VPC from other VPCs in your environment.

There may be cases where you may have VPCs other than the connected VPC in your SDDC, or SDDCs that needs to interact with native AWS services. To address this use case, VMware and AWS jointly engineered a new feature called VMware Transit Connect (powered by AWS Transit Gateway) within the VMware Cloud on AWS SDDC.

AWS Transit Gateway connects VPCs and on-premises networks through a central hub. It simplifies networking and puts an end to complex peering relationship. It acts as a cloud router.

VMware Transit Connect is an AWS Transit Gateway managed by VMware to allow customers to seamlessly interconnect multiple SDDC clusters, VPCs, and on-premises networks through a new construct called SDDC Groups.

Customers can now attach multiple SDDC clusters within an SDDC Group to the VMware Transit Connect or Managed Transit Gateway. It allows native Amazon VPCs to the attached to VMware Transit Connect, which greatly simplifies network connectivity while achieving high throughput, low latency, and high levels of resiliency.

Figure 4 – SDDC, on-premises, and VPC connectivity with VMware Transit Connect.

The architecture from Figure 4 allows customers to have a simplified multi-environment connectivity at scale. Customers who use VMware Transit Connect can now send traffic between all the constructs attached to the VMware Transit Connect.

Following is a description of how it works:

• On-Premises to SDDC: Customers can use AWS Direct Connect and Direct Connect Gateway to connect to VMware Transit Connect. With this model, SDDCs within an SDDC group can communicate to resources on-premises.
• SDDC to VPC: It’s also possible to attach an existing or new native Amazon VPC to VMware Transit Connect. This allows network routing between any SDDC within the SDDC Group and any VPC resources within the attached VPC.
• SDDC to SDDC: This model allows SDDCs within an SDDC that’s attached to VMware Transit Connect to communicate with each other.

Customers also have the option to use another VPC other than the connected VPC. They can attach this to the VMware Transit Connect and integrate that with VMware Cloud on AWS using the VMware Transit Connect. This allows workloads from other SDDCs to consume native services within the VPCs.

When a connected VPC is attached to a Transit Connect, VMware Cloud on AWS will prioritize traffic over the ENI to the SDDC over traffic path over the Transit Connect, and will automatically add routes to the main route table of the connected VPC.

The traffic flow patterns are currently supported when using the VMware Transit Connect, namely:

• Traffic flow between an SDDC to another SDDC within an SDDC Group.
• Traffic flow between an SDDC and native Amazon VPC including the connected VPC.
• Traffic flow between on-premises and SDDCs within an SDDC Group.

As you can see in Figure 4 above, customers have the flexibility to integrate native AWS services in other VPCs in the same or different AWS accounts with VMware Cloud on AWS environments within an SDDC group.

Depending on your use case, you can have a one-to-one mapping of the connected VPC to the SDDC in a simple deployment model, or you can extend it to a service provider model through the use of AWS PrivateLink, and finally leverage VMware Transit Connect for advanced deployments.

The purpose of the connected VPC is to allow customers to use native AWS services with VMware Cloud on AWS. For this reason, you may have to use the connected VPC or another VPC that can still communicate to VMware Cloud on AWS in order to use native AWS services.

Conclusion

When VMware Cloud on AWS is connected to the connected VPC, it always uses the default VPC route table in the VPC.

When using the connected VPC as a shared VPC or service provider VPC to other VPCs as well as VMware Cloud on AWS, we recommend the creation of a separate route table for traffic to other VPCs. You can associate the respective subnets to the route table.

The default route table should be dedicated to VMware Cloud on AWS. Where applicable, the route table is updated by VMware with new networks created within the SDDC.

There are scenarios where customers create separate route tables within a VPC for different reasons. In such a case, it’s worth remembering the custom route table will not be automatically updated when the main route table is updated based on any network changes within the SDDC. You’ll have to manually update the custom route table in the connected VPC.

Additionally, the connected VPC should not be used for transitive routing between other VPCs and VMware Cloud on AWS either via AWS Site-to-Site VPN or through third-party VPN appliances deployed within the connected VPC. This is not supported.

In this post, I have explained the AWS account structure in connection to VMware Cloud on AWS, and also explained things to consider when planning account setup for your connected VPC. I have also provided useful considerations around the selection and use of the connected VPC in relation to other VPCs customers may have within their environment.

Please reach out to us at AWS for support in implementing any of these architectures within your environments.