How to Enhance the Security and Compliance of Cloud Architectures with Datacom and AWS Services

By Bilal Shuja, Cloud Platform Specialist at Datacom
By Rishi Singla, Sr. Technical Account Manager at AWS

Security and compliance governance is one of the most challenging problems organizations face when managing their cloud infrastructure. According to Allianz Risk Barometer 2020, for the first time, cyber incidents rank as the most important business risk globally.

Unfortunately, new cloud security practices to deal with these risks are often shelved because of time constraints, lack of skilled resources, and knowledge gaps. Hence, before expanding cloud infrastructures and services, it’s important to evaluate not only the benefits, but the risks.

Datacom Group provides migration, transformation, and managed services across Australia and New Zealand for public clouds. After years of working with Amazon Web Services (AWS) cloud infrastructures, Datacom has observed that each client has their own industry-specific security and compliance requirements. What’s needed is a solution that is flexible enough to cater to diverse customer requirements.

Moreover, organizations are migrating to a multi-account environment, which makes the task of ensuring security and compliance even more challenging.

Variations in requirements come naturally with a multi-account structure. For instance, a development account usually requires less strict controls to allow developers the flexibility to experiment. A production account, on the other hand, must be fully secure and compliant per industry standards and best practices. And, as the number of accounts grows, governance becomes complex.

Datacom Group is an AWS Premier Consulting Partner and Managed Service Provider. They have AWS Competencies in SAP and Migration. In this post, we will present the architecture of the governance solution developed at Datacom Group to tackle the security and compliance challenges of a cloud infrastructure.

Solution Architecture

AWS already offers quite a few services for security assurance and compliance monitoring. Each of these services comes with its own benefits and advantages.

Datacom has evaluated the key features of native AWS services and selectively integrated them to offer a solution suitable for a multi-account and multi-environment (development, test, production) cloud infrastructure.

The Datacom governance solution monitors and audits the resources in multiple accounts from a central account, and gives you the option to automatically remediate any non-compliant resources. In addition, its aggregated reports summarize compliance status and security threats across all accounts.

Datacom’s solution has a primary-replica architecture in which a central Management account acts as the primary, and multiple Service accounts act as replicas.

Figure 1 – Solution architecture.

Amazon GuardDuty and AWS Config services in the Service accounts are the source of security and compliance events. When AWS Config detects a non-compliant resource, it invokes an AWS Lambda function to remediate the issue and make the resource compliant again. The solution includes a retry mechanism to handle failures. There’s also a possibility to manually invoke a Lambda function to execute remediation actions if required.

You can integrate the AWS Personal Health Dashboard to monitor the health of cloud-specific services relevant to a particular account. Events generated by AWS services are captured by AWS CloudWatch event rules. Using Amazon Simple Notification Service (SNS), these events are forwarded to the Management account. Remediation status and log messages for each action are also forwarded to the Management account via the same mechanism.

A Lambda function in the Management account receives notifications from different AWS services and accounts, and processes these notifications to generate reports. In addition, the same Lambda function in the Management account can forward critical notifications to IT service management (ITSM) tools, in case manual investigation is required.

The AWS Config service in the Management account also accumulates compliance information from multiple Service accounts, and provides an aggregated view for convenient monitoring of compliance state for all resources.

Why This Solution?

Datacom has a number of customers who have multi-account AWS environments. Manually provisioning resources for each customer is not only time-consuming, but also prone to human error. Providing management support for these resources is an operational challenge, which can lead to a degraded quality of service and unpleasant experience for customers.

To provide a swift and quality service, infrastructure as code (IaC) and automation are at the core of all the cloud solutions provided by Datacom.

Using a code-based implementation and DevOps methodologies for Datacom’s governance solution was a natural choice. It ensures an efficient delivery lifecycle, in which the solution is continuously developed and continuously tested. This results in a faster time to production without compromising quality.

Automation also reduces the deployment and provisioning timeframes from days to minutes. Scalability and non-consistent deployments are no longer a concern due to scripted deployment pipelines. All changes are tracked using Git source control, and proper versioning is maintained, which helps with traceability and allows for rollbacks in case of unexpected scenarios.

Prerequisites and Deployment Strategy

Code-based implementation is not only used for infrastructure provisioning but also for the development of AWS Config rules. The Datacom solution makes use of AWS Config Rule Development Kit (RDK) to develop custom and AWS-managed AWS Config rules. The AWS Config RDK also makes it possible to create and deploy account specific rules, tackling the challenge of varying compliance requirements for each account.

Similarly, the solution offers remediation actions that can be tailored to meet account- or customer-specific requirements.

The Datacom solution uses AWS DevOps tools to automate deployment. An AWS CodeBuild project in the Datacom-managed centralized CI/CD account orchestrates the solution deployment to customer-owned Management and Service accounts.

For deployment to work, cross account roles are essential. Two cross account roles are required. Let’s call them CrossAccountRoleA and CrossAccountRoleB.

Figure 2 – Deployment process.

CrossAccountRoleA needs to exist in the customer’s Management account and should have a trust relationship with the CI/CD account. Similarly, CrossAccountRoleB needs to exist in all customer Service accounts and should have trust relationships with the Management account.

The AWS CodeBuild project in the CI/CD account pulls the source code from the Datacom-owned Bitbucket Git repository, and then assumes the CrossAccountRoleA to create the following resources and services in the Management account:

• Amazon Simple Storage Service (Amazon S3) buckets.
• AWS Code Pipeline project.
• AWS Lambda functions.
• AWS Config service.

One of the Amazon S3 buckets in the Management account acts as the source for the AWS CodePipeline project. The pipeline has two stages:

• First is the Build stage that generates account-specific AWS CloudFormation templates for deploying AWS Config rules.
• Second is the Deploy stage that assumes the CrossAccountRoleB to create the CloudFormation stacks in the Service accounts. In addition to AWS Config Rules, the Deploy stage also provisions the following resources in the service accounts:
  • AWS CloudWatch event rules.
  • AWS Lambda functions for auto remediation.
  • Amazon Simple Query Service (SQS).
  • Amazon SNS.

Datacom’s governance solution takes approximately 10 minutes for an end-to-end deployment in a multi-account customer environment.

Once all of the resources are successfully deployed in the Service accounts, the AWS Config service triggers the compliance evaluation process. Events caused by AWS Config, Amazon GuardDuty, and the AWS Personal Health Dashboard are captured by CloudWatch event rules.

Amazon GuardDuty findings and Health Dashboard notifications are directly forwarded to a notification processor Lambda function in the Management account via SNS.

The compliance change notifications trigger the auto-remediation solution that first checks if a remediation action is defined for the underlying non-compliant AWS Config rule. If the respective remediation action exists, the resource is automatically remediated and the remediation status is sent to the Management account for reporting purposes.

If the remediation action is not defined for the respective AWS Config rule, details of the non-compliant resources and rules are forwarded to the Management account for further analysis by a security analyst.

The Lambda function in the Management account can be configured to create an ITSM ticket for manual remediation of the issue.

Conclusion

Datacom Group has already deployed its governance solution in a few customer environments and received a very positive feedback. In particular, auto-remediation functionality and scheduled consolidated reports reduced the operational overhead, both for Datacom and their customers.

In addition to the encouraging response, Datacom received further requirements for supporting more AWS Config rules and functionality for sending scheduled reports via email. Work is already in progress for integrating Amazon Simple Email Service (SES) with the Datacom solution, which will enable sending scheduled reports to the subscribers.

Datacom’s governance solution is flexible and can integrate with a number of AWS native services to offer enhanced capabilities. For example, Amazon Macie can be easily added to the solution to deal with data security and privacy issues. Amazon Athena can be used for analyzing reports stored in Amazon S3 and Amazon QuickSight to provide interactive dashboards.

The Datacom solution acts as a framework that can be tailored to a diverse range of functionalities to meet the needs of individual customers.

.

.

Datacom Group – AWS Partner Spotlight

Datacom Group is an AWS Premier Consulting Partner and MSP that provides migration, transformation, and managed services across Australia and New Zealand for public clouds.

Contact Datacom Group | Practice Overview

*Already worked with Datacom Group? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.