reference deployment

Cribl Stream for AWS CloudTrail

Deploy a vendor-neutral observability pipeline with AWS CloudTrail logging

View deployment guide

This Partner Solution deploys Cribl Stream to the Amazon Web Services (AWS) Cloud. Cribl is for DevOps users who want to deploy a vendor-neutral observability pipeline to route and enrich machine data.

Cribl Stream ingests data from many sources, such as Amazon Kinesis and Amazon Simple Queue Service (Amazon SQS). It organizes your data before routing it to other services for storage and analytics. It can write a full-fidelity retention copy of your data to object storage like Amazon Simple Storage Service (Amazon S3) and replay the data. Stream can also deliver your data to one or more analytics tools and process it before it reaches its destination. For example, if the data is security related, Stream can map it against a threat list.

This Partner Solution features Cribl Stream with AWS CloudTrail to log API calls to your Cribl instances and send alerts when it creates logs. For more information, refer to Cribl.

Cribl logo

This Partner Solution was developed by Cribl in collaboration with AWS. Cribl is an AWS Partner.

  •  What you'll build

  • The Partner Solution sets up the following:

    • A highly available architecture that spans two Availability Zones in your virtual private cloud (VPC).
    • An Application Load Balancer to route Cribl Stream user traffic to Cribl Stream instances in the public subnets.
    • In the public subnets, Cribl Stream deployed to Amazon Elastic Compute Cloud (Amazon EC2) instances in an Auto Scaling group.*
    • Amazon S3 for two buckets: one for Cribl Stream test data and another for CloudTrail logs.
    • CloudTrail to log API calls to Cribl instances to an S3 bucket and send logging notifications to Amazon Simple Queue Service (Amazon SQS).
    • Amazon SQS to notify subscribers when logs are written to the CloudTrail S3 bucket.
    • AWS Lambda to empty the CloudTrail data S3 bucket upon stack deletion.
    • AWS Identity and Access Management (IAM) to provide the following:
      • Lambda function and CloudTrail access to the CloudTrail data S3 bucket.
      • Cribl Stream instances access to the Cribl Stream test data S3 bucket. You can edit the Cribl Stream IAM policy after deployment to add other Cribl-supported data sources and destinations.

    * By default, to give users a better experience when getting started with Cribl Stream, this Partner Solution deploys to a public subnet. If you're deploying this Partner Solution to a production environment, consider using a private subnet.

  •  How to deploy

  • To deploy this Partner Solution, follow the instructions in the deployment guide, which includes these steps.

    1. Sign in to your AWS account. If you don't have an account, sign up at https://aws.amazon.com.
    2. Subscribe to Cribl Stream Single Instance (Free) x86_64 on AWS Marketplace.
    3. Launch the Partner Solution. The stack takes about 15 minutes to deploy. Before you create the stack, choose the AWS Region from the top toolbar. You can choose from the following options:
    4. Complete postdeployment steps in the deployment guide.

    Amazon may share user-deployment information with the AWS Partner that collaborated with AWS on this solution.  

    View deployment guide for details
  •  Cost and licenses

  • This Partner Solution requires a subscription to the Amazon Machine Image (AMI) for Cribl Stream Single Instance (Free) x86_64, available on AWS Marketplace.

    You are responsible for the cost of the AWS services and any third-party licenses used while running this solution. There is no additional cost for using the solution.

    This solution includes configuration parameters that you can customize. Some of these settings, such as instance type, affect the cost of deployment. For cost estimates, refer to the pricing pages for each AWS service you use. Prices are subject to change.

    Tip: After you deploy a solution, create AWS Cost and Usage Reports to track associated costs. These reports deliver billing metrics to an Amazon Simple Storage Service (Amazon S3) bucket in your account. They provide cost estimates based on usage throughout each month and aggregate the data at the end of the month. For more information, refer to What are AWS Cost and Usage Reports?