We use essential cookies and similar tools that are necessary to provide our site and services. We use performance cookies to collect anonymous statistics, so we can understand how customers use our site and make improvements. Essential cookies cannot be deactivated, but you can choose “Customize” or “Decline” to decline performance cookies.
If you agree, AWS and approved third parties will also use cookies to provide useful site features, remember your preferences, and display relevant content, including relevant advertising. To accept or decline all non-essential cookies, choose “Accept” or “Decline.” To make more detailed choices, choose “Customize.”
Customize cookie preferences
We use cookies and similar tools (collectively, "cookies") for the following purposes.
Essential
Essential cookies are necessary to provide our site and services and cannot be deactivated. They are usually set in response to your actions on the site, such as setting your privacy preferences, signing in, or filling in forms.
Performance
Performance cookies provide anonymous statistics about how customers navigate our site so we can improve site experience and performance. Approved third parties may perform analytics on our behalf, but they cannot use the data for their own purposes.
Allowed
Functional
Functional cookies help us provide useful site features, remember your preferences, and display relevant content. Approved third parties may set these cookies to provide certain site features. If you do not allow these cookies, then some or all of these services may not function properly.
Allowed
Advertising
Advertising cookies may be set through our site by us or our advertising partners and help us deliver relevant marketing content. If you do not allow these cookies, you will experience less relevant advertising.
Allowed
Blocking some types of cookies may impact your experience of our sites. You may review and change your choices at any time by selecting Cookie preferences in the footer of this site. We and selected third-parties use cookies or similar technologies as specified in the AWS Cookie Notice.
Your privacy choices
We display ads relevant to your interests on AWS sites and on other properties, including cross-context behavioral advertising. Cross-context behavioral advertising uses data from one site or app to advertise to you on a different company’s site or app.
To not allow AWS cross-context behavioral advertising based on cookies or similar technologies, select “Don't allow” and “Save privacy choices” below, or visit an AWS site with a legally-recognized decline signal enabled, such as the Global Privacy Control. If you delete your cookies or visit this site from a different browser or device, you will need to make your selection again. For more information about cookies and how we use them, please read our AWS Cookie Notice.
Centralized Logging with OpenSearch helps organizations collect, ingest, and visualize log data from various sources using Amazon OpenSearch Service. This AWS Solution provides a web-based console, which you can use to create log ingestion pipelines with a few clicks. Log ingestion pipelines include log collection agent deployment, log enrichment without writing codes, buffer layer creation, and OpenSearch index configuration. After logs are stored in OpenSearch Service, the solution automatically generates ready-to-use dashboards for analyzing AWS service logs and application logs in different formats (for example, Nginx, JSON, and Spring Boot). In combination with other AWS services, this solution provides you with a turnkey environment to begin logging and monitoring your AWS applications.
Benefits
Ease of use
Use a web console from your AWS account to ingest both application and AWS service logs, then analyze the logs with visualization dashboards.
Improved operational efficiency
Serverless technologies with built-in high availability and a pay-for-use billing model reduces the need for infrastructure management, allowing you to focus more on building log analytics for your business.
Open source and customization
The solution is open sourced and free for commercial use. You pay only for the AWS usage. You can take the source code as a reference to make your own implementation that fits your needs.
Step 6 AWS Step Functions orchestrates on-demand CloudFormation deployment of a set of predefined stacks for log pipeline management. The log pipeline stacks deploy separate AWS resources and are used to collect and process logs and ingest them into Amazon OpenSearch Service for further analysis and visualization.
Step 8 AWS Systems Manager and Amazon EventBridge manage log agents for collecting logs from application servers, such as installing log agents (Fluent Bit) for application servers and monitoring the health status of the agents.
Step 2 Amazon Cognito user pool or OpenID Connector (OIDC) can be used for authentication.
Step 3 AWS AppSync provides the backend GraphQL APIs.
Step 4 Amazon DynamoDB stores the solution-related information as a backend database.
Step 5 AWS Lambda interacts with other AWS services to process core logic of managing log pipelines or log agents and obtains information updated in DynamoDB tables.
Step 6 AWS Step Functions orchestrates on-demand CloudFormation deployment of a set of predefined stacks for log pipeline management. The log pipeline stacks deploy separate AWS resources and are used to collect and process logs and ingest them into Amazon OpenSearch Service for further analysis and visualization.
Step 8 AWS Systems Manager and Amazon EventBridge manage log agents for collecting logs from application servers, such as installing log agents (Fluent Bit) for application servers and monitoring the health status of the agents.
Step 2 Amazon Cognito user pool or OpenID Connector (OIDC) can be used for authentication.
Step 3 AWS AppSync provides the backend GraphQL APIs.
Step 4 Amazon DynamoDB stores the solution-related information as a backend database.
Step 5 AWS Lambda interacts with other AWS services to process core logic of managing log pipelines or log agents and obtains information updated in DynamoDB tables.
Step 6 AWS Step Functions orchestrates on-demand CloudFormation deployment of a set of predefined stacks for log pipeline management. The log pipeline stacks deploy separate AWS resources and are used to collect and process logs and ingest them into Amazon OpenSearch Service for further analysis and visualization.
Step 6 AWS Step Functions orchestrates on-demand CloudFormation deployment of a set of predefined stacks for log pipeline management. The log pipeline stacks deploy separate AWS resources and are used to collect and process logs and ingest them into Amazon OpenSearch Service for further analysis and visualization.
Step 8 AWS Systems Manager and Amazon EventBridge manage log agents for collecting logs from application servers, such as installing log agents (Fluent Bit) for application servers and monitoring the health status of the agents.
Step 2 Amazon Cognito user pool or OpenID Connector (OIDC) can be used for authentication.
Step 3 AWS AppSync provides the backend GraphQL APIs.
Step 4 Amazon DynamoDB stores the solution-related information as a backend database.
Step 5 AWS Lambda interacts with other AWS services to process core logic of managing log pipelines or log agents and obtains information updated in DynamoDB tables.
Step 6 AWS Step Functions orchestrates on-demand CloudFormation deployment of a set of predefined stacks for log pipeline management. The log pipeline stacks deploy separate AWS resources and are used to collect and process logs and ingest them into Amazon OpenSearch Service for further analysis and visualization.
Step 8 AWS Systems Manager and Amazon EventBridge manage log agents for collecting logs from application servers, such as installing log agents (Fluent Bit) for application servers and monitoring the health status of the agents.
Step 2 Amazon Cognito user pool or OpenID Connector (OIDC) can be used for authentication.
Step 3 AWS AppSync provides the backend GraphQL APIs.
Step 4 Amazon DynamoDB stores the solution-related information as a backend database.
Step 5 AWS Lambda interacts with other AWS services to process core logic of managing log pipelines or log agents and obtains information updated in DynamoDB tables.
Step 6 AWS Step Functions orchestrates on-demand CloudFormation deployment of a set of predefined stacks for log pipeline management. The log pipeline stacks deploy separate AWS resources and are used to collect and process logs and ingest them into Amazon OpenSearch Service for further analysis and visualization.
Related content
Solution Web Console
This image shows a preview of the web console for Centralized Logging with OpenSearch.
This blog describes an Amazon CloudWatch capability to search, analyze, and correlate cross-account telemetry data stored in CloudWatch such as metrics, logs, and traces.