Posted On: Feb 27, 2024
Amazon Data Firehose (Firehose) decompression for CloudWatch Logs now supports message extraction, so customers can automatically filter out header information and deliver only the message content from their CloudWatch logs to destinations such as Amazon S3 and Splunk for analytics.
Customers use Firehose to decompress, aggregate, and deliver log events from their applications and services captured in Amazon CloudWatch Logs to destinations such as Amazon S3 and Splunk for use cases such as application troubleshooting and audit compliance. CloudWatch log records use a nested JSON structure, and the message in each record is embedded within header information. Many customers want to filter out the header information and simply deliver the embedded message to the destination, because it reduces the cost of subsequent processing and storage. With message extraction, Firehose provides a simple option that customers can select to filter out the header information and only deliver the embedded message content. There is no additional charge to apply message extraction when customers use Firehose decompression for CloudWatch Logs.
To learn more about the feature and get started, visit Amazon Data Firehose documentation and console.