AWS Marketplace

Using Cribl Stream on AWS Control Tower with Account Factory Customization

In this post, Kam, Michelle, and I show you how to deploy the Cribl Stream observability Partner Solution into a new AWS account as part of the provisioning process using Account Factory Customization.

Cribl Stream is a vendor-agnostic observability pipeline that gives you the flexibility to collect, reduce, enrich, normalize, and route data from any source to any destination within your existing data infrastructure. This deployment of Cribl Stream has been packaged into an AWS Partner Solution. AWS Partner Solutions are automated reference deployments for key workloads on the AWS Cloud. Each Partner Solution launches, configures, and runs the AWS compute, network, storage, and other services required to deploy a specific workload on AWS, using AWS best practices for security and availability.

AWS Control Tower offers the easiest way to set up and govern a secure, multi-account AWS environment. It establishes a landing zone that is based on best-practices blueprints, and it enables governance using guardrails you can choose from a prepackaged list. The landing zone is a well-architected, multi-account baseline that follows AWS best practices. Guardrails implement governance rules for security, compliance, and operations.

Account Factory Customization is an enhancement to AWS Control Tower that enables the customization of AWS accounts during the account provisioning process. With Account Factory Customization, you have access to predefined customization blueprints for specific use cases. The blueprints can be Partner-managed, customer-managed, or in future phases, AWS-managed.

Prerequisites

Cribl Stream requires an AWS Marketplace subscription. For this deployment, you can use the no-cost subscription Cribl Stream Single Instance (Free) x86_64.

You must verify that your AWS Service Catalog has been properly configured. If you need help getting started, refer to the AWS Service Catalog Administrator Guide.

Solution Overview

The Cribl Stream Partner Solution deploys the following resources:

  • A highly available architecture that spans two Availability Zones in your virtual private cloud (VPC).
  • An Application Load Balancer to route traffic to Cribl Stream instances.
  • In the public subnets, Cribl Stream deployed to Amazon Elastic Compute Cloud (Amazon EC2) instances in an Auto Scaling group.
  • An Amazon Simple Storage Service (Amazon S3) bucket to test sending and receiving Cribl Stream data.
  • AWS Identity and Access Management (IAM) for a role and policy providing the Cribl Stream instances access to the S3 bucket. You can add other Cribl-supported data sources and destinations by editing the IAM policy after deployment.
  • By default, to give users a better experience when getting started with Cribl Stream, this solution deploys in a public subnet. If you’re deploying this solution in a production environment, consider using a private subnet. Refer to the following diagram.

Cribl Stream AWS Control Tower Account Factory Customization Architecture Diagram

Solution walkthrough: Using Cribl Stream for observability on AWS Control Tower with Account Factory Customization

In this walkthrough, we will show you how to add Cribl Stream to your AWS Service Catalog portfolio and then deploy Cribl Stream into a new account using Account Factory Customization.

1. Adding Cribl Stream to an AWS Service Catalog portfolio

To add Cribl Stream to your AWS Service Catalog portfolio of your management account, do the following:

  1. Access the AWS Service Catalog console and ensure AWS Service Catalog has been configured.
  2. On the left panel under the Administration subheading, choose the Getting Started Library.
  3. In the Getting Started Library, on the right panel, under the Products section, use the Search products field and enter Cribl Stream. Once you have located Cribl Stream, on the left, select the option indicator. The Add to portfolio button will become enabled, enabling you to proceed to the next step. There are several methods for creating a portfolio; in this blog post, we will create the portfolio once we have selected the product.
  4. Select the Add to portfolio button. This opens the Add to portfolio: Cribl Stream page. In the bottom half of the page, under the Portfolio details section, select Create a new portfolio.
  5. Enter the appropriate values into the Portfolio name, Portfolio description and Owner fields and select Add to portfolio.

A success indicator will confirm that Cribl Stream has been added to the portfolio, and you will be taken to the page for the newly created portfolio.

2. Using AWS Control Tower to provision your new portfolio

To use AWS Control Tower to provision your new portfolio, do the following.

  1. Go to the AWS Control Tower console of your Management account. In the left sidebar, navigate to and select Account Factory.
  2. On the right panel in the Account Factory page, select Create Account. This opens the Create Account page.
  3. In the top area of the Create Account page, within the Account details section, enter the correct information in the fields for Account Email, Display name, Identity Center user email, IAM Identity Center user name (First name and Last name), and Organizational unit.

3. Using Account Factory Customization

To customize your new account, do the following:

  1. In the bottom area of the Create Account page, within the Account Factory customization section, enter your AWS Service Catalog hub account number in the Account that contains your AWS Service Catalog products field. This may be the same account you are currently on. Once the account is entered, you will be able to choose the product you imported into the Getting Started Library in step 1.4.
  2. In the Select a product that already exists field, select Cribl Stream from the dropdown. This will cause the next field, Product version, to be populated with the latest version of the product from AWS Service Catalog.
  3. The next section Blueprint parameters lists all the parameters required to deploy the product, along with their descriptions and default values if default values are available. In this case, all the parameters required to deploy Cribl Stream are as follows:
    1. Availability Zones
    2. VPC CIDR
    3. Public subnet 1 CIDR
    4. Public subnet 2 CIDR
    5. Private subnet 1 CIDR
    6. Private subnet 2 CIDR
    7. VPC tenancy
  4. The Deployment Regions defines the Regions where you are deploying Cribl Stream. Home Region is the Region where you access the AWS Management Console. All governed Regions refers to all Regions being managed by your AWS Control Tower environment. Choose Home Region.
  5. You are now ready to create the new account with Cribl Stream. Select Create Account. This process can take a few minutes, and depending on the product you are deploying, that time may increase.

4. Logging into the new account

With the deployment complete, you are ready to access the newly provisioned account. To do that, do the following:

  1. In the AWS Control Tower console left panel, select the Users and access option.
  2. On the Users and access page, select the User portal URL. This launches the portal sign in page in a separate browser tab or window. On that tab or window, proceed with the Sign in process.
  3. Upon successful sign in, you should see a page listing all accounts. Select your newly created account.
  4. The account section expands to reveal the access roles for the account.
  5. Select Management Console.

5. Log into Cribl Stream

  1. At the top of the Management Console, use the Search bar to locate and access the CloudFormation Console.
  2. In the left panel, select Stacks. On the right panel, all deployed Stacks will be listed. Select the Stack name of the CriblDeploy Partner Solution. This opens the CriblDeploy stack page.
  3. In the top section of the CriblDeploy stack page, a series of tabs will denote different sections of the stack.
  4. Select the Ouputs tab. The outputs section will list the Default Web Access Credentials and the Cribl LogsCribl Stream Web URL.

Conclusion

In this post, Kam, Michelle, and I showed you how to improve observability by deploying the Cribl Stream Partner Solution into a new AWS account during the provisioning process. This solution uses Account Factory Customization, which enables you to create custom accounts natively in AWS Control Tower. It also provides flexibility to define custom resources and configurations and reduces account maintenance overhead. We encourage you to learn how Cribl Stream helps manage your observability content and to visit the AWS Partner Solutions portal to learn about all the different products available.

Cleanup

To remove Cribl Stream from the newly created account, access the new account and go to the CloudFormation Console. Once there, select the Cribl Stream main stack and select Delete Stack.

If you would rather delete the account created with Account Factory, visit the Close an account created in Account Factory for details.

Next steps

To find out more about the Cribl Stream Partner Solution, visit Cribl Stream on AWS. To find out more about Cribl solutions available in AWS Marketplace, visit the Cribl seller page in AWS Marketplace.

About the authors

Tony Bulding is a Partner Solutions Architect with the Integration and Automation team. He focuses on helping partners automate their product deployments on AWS and works with partners that want to Build on AWS Control Tower.

Kam Amir is the Director of Technical Alliances at Cribl. He builds that partnerships with sources, destinations and enrichment technologies that help customers adopt the right solutions for their businesses and get the most value out of their observability data.

Michelle Zhang is a Solutions Marketing Manager at Cribl focused on strategizing and delivering joint marketing programs with AWS. She helps customers see the value of Cribl and AWS solutions.