AWS Partner Network (APN) Blog
Enhance Data Visibility with Cribl Search and Amazon Managed Grafana
By: Sunil Ramachandra, Sr Solutions Architect – AWS
By: Aswin Vasudevan, Sr Solutions Architect – AWS
By: Rizwan Mushtaq, Principal Solutions Architect – AWS
By: Kamilo “Kam” Amir, Director Business Development – Cribl
 |
 |
In today’s digital landscape, organizations face challenges when managing the growth of operational data across their infrastructure. The high volume of logs, metrics, and traces generated by modern applications and systems creates both opportunities and operational complexity for IT teams. This information holds the key to deeper insights and improved performance, but organizations need a scalable, customizable observability pipeline to efficiently collect, process, and route this data to their preferred destinations.
Cribl is an AWS Partner Network (APN) partner that delivers centralized data management and configurable routing capabilities for rapidly growing volumes of operational and security data. Cribl collaborates closely with AWS to provide organizations with enhanced security and compliance controls while maintaining the freedom to choose best-fit tools for their observability needs. Once data is processed within the Cribl platform, Amazon Managed Grafana serves as a robust visualization engine, transforming this processed data into actionable insights through intuitive dashboards and real-time monitoring capabilities.
In this post, we’ll explore how integrating Cribl Search with Amazon Managed Grafana’s capabilities unlocks a new level of operational intelligence. Whether you’re managing a sprawling cloud infrastructure or fine-tuning a single application, this integration empowers you to make data-driven decisions faster, with greater confidence, and at scale.
Figure 1: Cribl Search integration with Amazon Managed Grafana
The ability to integrate with Amazon Managed Grafana enhances visualization and search capability across the organization. As shown in Figure 1, Cribl Search operates across several data sources such as Amazon S3, Cribl Lake, Amazon Security Lake and other native AWS services exposed via AWS APIs. For full set of data sources supported by Cribl search, refer to Cribl Search documentation. It is useful for the following use-cases:
1. Cloud Infrastructure Monitoring
As shown in Figure 1, Cribl Search can query via APIs exposed by AWS Services or other data sources at rest, without having to index this data first. This allows users to search and find only the events that are relevant and send them with context through Cribl Stream to other Security Information and Event Management (SIEM) systems for analysis. This integration enables teams to create real-time dashboards for AWS resource utilization, cost analysis and performance metrics. Amazon Managed Grafana then visualizes this data through customizable dashboards, showing infrastructure health across AWS Regions and services
2. Application Performance Management
Within Amazon Managed Grafana, teams can create application-specific dashboards showing request latency, error rates, and user experience metrics, with drill-down capabilities for detailed transaction analysis.
3. Security Operations
Integration with Amazon Managed Grafana displays security events in dedicated security operations dashboards, enabling rapid incident response and investigation. Cribl platform enables continuous security event monitoring, streamlines compliance reporting, and enhances threat detection and investigation workflows. SOC teams benefit from optimized operational procedures and faster response times.
Prerequisites
To perform the solution, you need to have the following prerequisites in place:
- An AWS account with administrative access
- An existing Amazon S3 bucket that VPC Flow Logs has permission to write to or create a new S3 bucket
- Appropriate Identity and Access Management (IAM) permissions configured for the user who need to enable flow logs
- A Cribl Cloud account
Solution Walkthrough
In this setup, you’ll set up authentication using API Tokens Keys to secure communication between Cribl and Amazon Managed Grafana. You’ll then install and configure the Cribl search plugin in the Amazon Managed Grafana console, creating a seamless connection. Next, you’ll create visualizations using VPC Flow Logs data with simple query parameters. Finally, you’ll configure advanced table search for detailed network analysis, enabling comprehensive system monitoring and troubleshooting within a unified observability pipeline.
1. Set Up API Authentication
Security is key in this process, you’ll use API tokens to secure the authentication between Cribl and Amazon Managed Grafana. To access the API Credentials section in Cribl, log into the Cribl admin dashboard, select your organization, and navigate to the sidebar menu where “API Credentials” is listed as per Figure 2. This section allows management of Client IDs, secrets, and credential settings for integrations.
Figure 2: Gathering API credentials from Cribl Organization
2. Install and Configure the Plugin
Open Amazon Managed Grafana and navigate to the Plugins section. Navigate to ‘Add new connection’ as per Figure 3 and search for “Cribl” in the plugins section to Add Connection
Figure 3: Setting up Cribl Search Plugin within Amazon Managed Grafana console
Enter your Cribl credentials to establish a secure connection with Amazon Managed Grafana as per Figure 4.
Figure 4: Establishing secure connection with Cribl Search from Amazon Managed Grafana
3. Create Your First Visualization
To begin creating your first visualization, click on the Query tab which is highlighted as per Figure 5. You’ll see a large text input area – this is your query editor. Now, let’s add some powerful search parameters.
Copy the query below and paste it directly into the query editor. This search will pull VPC Flow Logs data from the last 15 minutes, grouping it by log status in one-minute intervals. Once you’ve added the query, you’re ready to run it and start building your visualization. Refer to the documentation for details.
dataset="cribl_search_sample" dataSource="vpcflowlogs" earliest=-15m
Figure 5: Cribl search Query dashboard for VPC Flow Logs
4. Dive Deeper with Table Search inside Amazon Managed Grafana
You can use Table Search inside Amazon Managed Grafana to search through individual log entries, pinpoint system anomalies, and trace network request paths across your services. Switch from time series graphs to table views in one click, all within Amazon Managed Grafana’s interface. The query below is a sample that pulls 1000 entries from the dataset. By looking at the various IPs, Security Operation Center (SOC) analysts can perform a more detailed analysis. Further, this query can be customized to help speed up any investigation
Example: dataset="cribl_search_sample" | limit 1000
Cleanup and Cost Considerations
There is a cost associated with using this solution for the services (AWS and Cribl) involved. To avoid incurring unnecessary charges, follow these steps to clean up the resources you created during this walkthrough:
- Delete VPC Flogs Logs including the S3 bucket created as part of the solution walkthrough. After you delete a flow log, it can take several minutes to stop collecting data. Deleting a flow log doesn’t delete the log data from the destination or modify the destination resource. You must delete the existing flow log data directly from the destination and clean up the destination resource, using the console for the destination service.
- Remove any temporary or unused configurations used in Cribl.
Conclusion
The integration of Cribl and Amazon Managed Grafana delivers a customizable observability pipeline. Organizations benefit from centralized data management, improved security and compliance, configurable routing, and visualization—all while retaining the freedom to choose best-fit tools. This makes the Cribl integration with Amazon Managed Grafana valuable for enterprises seeking scalable, future-ready observability solutions.
Next Steps and Resources
Explore Amazon Managed Grafana: Start your journey with Amazon Managed Grafana. Here you’ll find comprehensive information about features, pricing, and enterprise capabilities.
Get Started with Cribl: Begin by subscribing to Cribl Cloud through the AWS Marketplace. Cribl’s extensive documentation offers integration guides and tutorials to accelerate your implementation. Take advantage of Cribl’s free education, certifications, sandbox environments, sample datasets and pre-built pipelines to jumpstart your Data Engine for IT and security journey
.
Cribl – AWS Partner Spotlight
Cribl is an AWS Advanced Technology Partner and AWS Competency Partner empowering organizations to transform their data strategy. Customers use Cribl to collect, process, route, and analyze all IT and security data, delivering the flexibility, choice, and control required to adapt to their ever-changing needs.