Posted On: Jun 21, 2022
Amazon ECS now fully supports multiline logging powered by AWS for Fluent Bit for both AWS Fargate and Amazon EC2. AWS Fluent Bit is an AWS distribution of the open-source project Fluent Bit, a fast and a lightweight log forwarder. Amazon ECS users can use this feature to re-combine partial log messages produced by your containerized applications running on AWS Fargate or Amazon EC2 into a single message for easier troubleshooting and analytics.
The best practice for containerized applications is to send logs to the standard output of the operational system, such as stdout or stderr. AWS Fargate container runtime splits long log messages exceeding 16KB max buffer size into partial messages for optimal performance results. As result, users can face challenges working with long application logs messages such as stack traces when they arrive at the final destination, like analytics solutions or logs storage.
AWS Fluent Bit now supports a multiline filter, a capability that helps concatenate partial log messages that originally belong to one context but were split across multiple records or log lines for both ECS EC2 and Fargate. Customers can use AWS for Fluent Bit to route logs from their containerized applications to AWS Services, such as Amazon CloudWatch and Amazon Kinesis Data Firehose or partner solutions for log analytics and storage. Amazon ECS customers can use FireLens to configure AWS for Fluent Bit or setup AWS for Fluent Bit as a sidecar or daemon manually.