Cribl is primarily used to reduce data volume. When large datasets arrive, such as 1 TB of data, it can be reduced by 600 GB or 400 GB while maintaining the same information. Additionally, Cribl is used to send the same data to multiple destinations. The same data can be copied and sent to different products such as Splunk and Dynatrace.
For firewall logs, there are many default parsing templates and pipelines available. Firewall logs can be easily converted using parser functions. Default parsers are available for all log types, such as Palo Alto traffic, access logs, audit logs, and Linux logs. When a parser function is chosen for Palo Alto traffic, it automatically extracts all fields from the firewall logs.
A specific use case implemented involves firewall logs, which are substantial in size. Statistics are performed on the firewall logs and sent every five minutes. The logs are summarized by state count, and during that five-minute interval, the logs are aggregated and sent to other locations such as Dynatrace and Splunk. This significantly reduces data size and saves considerable space and licensing costs in Splunk.
Cribl provides substantial help with sending data to different destinations. With three products in use—Splunk, Dynatrace, and DataDog—Cribl sends dual feeds to multiple products. For instance, firewall logs are needed by both Splunk and DataDog. Additionally, some observability logs are directed to Dynatrace while remaining logs are sent to Splunk. Cribl effectively splits data across the various products in use.
Cribl is recommended for organizations with more than 1 TB or 2 TB of data ingestion. For smaller data volumes of less than 1 TB, Splunk licensing alone is sufficient, and parsing can be done at the Splunk level. With 14 TB of data ingestion per day, Cribl provides significant benefits.