Build More Reliable and Secure Windows Services Using Amazon Kinesis Agent for Microsoft Windows

September 8, 2021: Amazon Elasticsearch Service has been renamed to Amazon OpenSearch Service. Visit the website to learn more.

We’ve all been there. You’ve deployed a new service on Windows servers. Maybe it’s based on Microsoft technology such as IIS, AD, DHCP, Microsoft Exchange, or SharePoint, or maybe it’s your own service running on Windows. Maybe the servers are on premises, or maybe they’re in the AWS cloud, or possibly you’ve got a hybrid situation. You’ve deployed the service, but you don’t have a good sense of how healthy the service is, or if there are any security issues with it. It seems to be working, but you’re worried that you just can’t give it enough attention until your customers complain.

What if you could do better than that? Without investing in expensive monitoring applications? Without having to write your own agent that pulls logs, events, and metrics and sends them to the cloud? Well, today you can! Introducing Amazon Kinesis Agent for Microsoft Windows: the agent that helps you be proactive instead of reactive for the services you build and operate.

Here are some really great things about Amazon Kinesis Agent for Microsoft Windows:

• It’s free. You just pay for the AWS services you use.
• It’s easy to install, configure, and use.
• It’s open sourced at awslabs/kinesis-agent-windows – you can see exactly what it does, and learn how to extend it through plugins.
• Amazon engineers are actively engaging with the open source community and want your feedback. Feel free to open issues – there will be a response! You can also engage with the Amazon Kinesis community on the Amazon Kinesis Forum.
• It’s very configurable without writing a line of code. A JSON-formatted configuration file describes:
  • What data you want to collect (logs, Windows event logs, ETW trace events, performance counters)
  • How you’d like to parse the data (optional)
  • How you’d like to filter the data (optional)
  • How you’d like to transform the data, for example to JSON or XML (optional)
  • Where you’d like to stream the data, for example to Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, Amazon CloudWatch Logs , or Amazon CloudWatch Metrics.
• It’s very reliable, even with network interruptions, log rotations, log locking, reboots, etc.
• It has a built-in cooperative throttling mechanism that keeps data flowing smoothly to the cloud.
• It’s designed to be deployed across large fleets of servers with automatic configuration and self-updating capabilities, all under your control.
• It can be deployed to fleets of Windows clients like desktops and laptops to give you end-to-end visibility of the customer experience.
• It’s fully documented (see the User Guide).

Once the data is up in the cloud, you can implement the exact data pipeline you need to get the results you want. You’re not tied to one particular service or endpoint. For example, you could:

• Stream JSON-formatted data to Amazon Kinesis Data Firehose that then stores the data in S3. You can then query S3 directly using Amazon Athena, or ETL into an Amazon Redshift data warehouse and visualize the data using Amazon QuickSight.
• Stream JSON-formatted data to Amazon Kinesis Data Firehose that then stores the data in Amazon Elasticsearch. You can then query using Kibana.
• Stream data to custom analytics applications via Amazon Kinesis Data Streams. The applications can be implemented serverless style using AWS Lambda or Amazon Kinesis Data Analytics, or using a server-based approach with Amazon Elastic Compute Cloud (Amazon EC2) instances.
• Stream metrics to Amazon CloudWatch. Then you can create dashboards and alarms.
• Stream logs to Amazon CloudWatch Logs and inspect the centralized logs in the AWS console.

Amazon Kinesis Agent for Microsoft Windows can save on operational costs, too. You can transform logs before streaming them to the cloud, which saves the expense of converting these logs in a second step. You can filter out the data you don’t need before it’s streamed, minimizing data transfer, API, and storage costs.

You can easily create your own plug-ins if needed to respond to any monitoring challenge, even complex situations.

Amazon Kinesis Agent for Microsoft Windows helps you gain better operational insight into the health and security of your services, enabling you to proactively and efficiently maintain your fleet of Windows servers.

Next steps:

• Learn how to Collect, parse, transform, and stream Windows events, logs, and metrics using Amazon Kinesis Agent for Microsoft Windows.
• Learn more about Amazon Kinesis Agent for Windows.
• Get all the agent details, tutorial, and configuration examples in the Amazon Kinesis Agent for Microsoft Windows User Guide.
• Read the Amazon Kinesis Documentation.
• Download the agent binaries: Amazon Kinesis Agent for Windows download
• Get the agent source code from awslabs/kinesis-agent-windows
• Let the community know about problems and opportunities at: submit issues.

We look forward to your feedback and participation!