AWS Compute Blog

Category: Analytics

Deploying Local Gateway Ingress Routing on AWS Outposts

This post is written by Leonardo Solano, Senior Hybrid Cloud Solution Architect and Chris Lunsford, Senior Specialist Solutions Architect, AWS Outposts. AWS Outposts lets customers use the same Amazon Virtual Private Cloud (VPC) security mechanisms, such as security groups and network access control lists, to control traffic flows for on-premises applications running on Outposts. Some […]

Reference architecture

Using custom consumer group ID support for AWS Lambda event sources for MSK and self-managed Kafka

This post shows how to use the new custom consumer group ID feature of the Lambda event source mapping for Amazon MSK and self-managed Kafka. This feature can be used to consume messages with Lambda starting at a specific timestamp or offset within a Kafka topic. It can also be used to consume messages from a consumer group that is replicated from another Kafka cluster using MirrorMaker v2.


Building a low-code speech “you know” counter using AWS Step Functions

This post is written by Doug Toppin, Software Development Engineer, and Kishore Dhamodaran, Solutions Architect. In public speaking, filler phrases can distract the audience and reduce the value and impact of what you are telling them. Reviewing recordings of presentations can be helpful to determine whether presenters are using filler phrases. Instead of manually reviewing […]

Orchestrating AWS Glue crawlers using AWS Step Functions

This blog post is written by Justin Callison, General Manager, AWS Workflow. Organizations generate terabytes of data every day in a variety of semistructured formats. AWS Glue and Amazon Athena can give you a simpler and more cost-effective way to analyze this data with no infrastructure to manage. AWS Glue crawlers identify the schema of […]

Batching controls with Lambda event source mapping

Introducing AWS Lambda batching controls for message broker services

This post is written by Mithun Mallick, Senior Specialist Solutions Architect. AWS Lambda now supports configuring a maximum batch window for instance-based message broker services to fine tune when Lambda invocations occur. This feature gives you an additional control on batching behavior when processing data. It applies to Amazon Managed Streaming for Apache Kafka (Amazon […]

Query large datasets

Visualizing AWS Step Functions workflows from the Amazon Athena console

Amazon Athena console now provides an integration with AWS Step Functions’ workflows. You can use the provided patterns to create and visualize Step Functions’ workflows directly from the Amazon Athena console. Step Functions’ workflows that use Athena’s optimized API integration appear in the Athena console. To learn more about Amazon Athena, read the user guide.

Offset lag metric for Amazon MSK as an event source for Lambda

This post written by Adam Wagner, Principal Serverless Solutions Architect. Last year, AWS announced support for Amazon Managed Streaming for Apache Kafka (MSK) and self-managed Apache Kafka clusters as event sources for AWS Lambda. Today, AWS adds a new OffsetLag metric to Lambda functions with MSK or self-managed Apache Kafka event sources. Offset in Apache […]

Consumer function log stream

Introducing mutual TLS authentication for Amazon MSK as an event source

This post is written by Uma Ramadoss, Senior Specialist Solutions Architect, Integration. Today, AWS Lambda is introducing mutual TLS (mTLS) authentication for Amazon Managed Streaming for Apache Kafka (Amazon MSK) and self-managed Kafka as an event source. Many customers use Amazon MSK for streaming data from multiple producers. Multiple subscribers can then consume the streaming […]