No servers to manage

Fully managed

With a few clicks in the console, you can create a fully managed Apache Kafka cluster that follows Apache Kafka’s deployment best practices, or create your own cluster using a custom configuration. Once you create your desired configuration, Amazon MSK automatically provisions, configures, and manages your Apache Kafka cluster operations and Apache ZooKeeper nodes.

Apache ZooKeeper included

Apache ZooKeeper is required to run Apache Kafka, coordinate cluster tasks, and maintain state for resources interacting with the cluster. Amazon MSK manages the Apache ZooKeeper nodes for you. Each Amazon MSK cluster includes the appropriate number of Apache ZooKeeper nodes for your Apache Kafka cluster at no additional cost.

Amazon MSK Serverless

MSK Serverless is a cluster type for Amazon MSK that makes it easy for you to run Apache Kafka clusters without having to manage compute and storage capacity. MSK Serverless automatically provisions and scales resources while also managing Apache Kafka partitions, so you can stream data without having to worry about right-sizing or scaling clusters.

Highly available

High availability is default

All clusters are distributed across multiple AZs (three is the default), are supported by Amazon MSK’s service-level agreement, and are supported by automated systems that detect and respond to issues within cluster infrastructure and Apache Kafka software. If a component fails, Amazon MSK automatically replaces it without downtime to your applications. Amazon MSK manages the availability of your Apache ZooKeeper nodes so you don’t need to start, stop, or directly access the nodes yourself. It also automatically deploys software patches as needed to keep your cluster up to date and running smoothly.

Data replication

Amazon MSK uses multi-AZ replication for high availability. Data replication is included at no additional cost.

Highly secure

Private connectivity

Your Apache Kafka clusters run in an Amazon Virtual Private Cloud (VPC) managed by Amazon MSK. Your clusters are available to your own Amazon VPCs, subnets, and security groups based on the configuration you specify. You have complete control of your network configuration and IP addresses.

Granular access control

IAM Access Control is a no-cost security option that simplifies cluster authentication and Apache Kafka API authorization using IAM roles or user policies to control access. Using IAM Access Control, you no longer need to build and run one-off access management systems to control client authentication and authorization for Apache Kafka. Your clusters are secured using least-privileged permissions by default. For provisioned clusters, you also can use Simple Authentication and Security Layer (SASL)/Salted Challenge Response Authentication Mechanism (SCRAM) or mutual Transport Layer Security (TLS) authentication with Apache Kafka access control lists (ACLs) to control client access.

Encryption at rest and in transit

Amazon MSK encrypts your data at rest without special configuration or third-party tools. For provisioned clusters, all data at rest can be encrypted using AWS Key Management Service (KMS) key by default or your own key. You can also encrypt data in transit via TLS between brokers and between clients and brokers on your cluster. For serverless clusters, all data at rest is encrypted by default using service-managed keys, and all data in transit is encrypted by default via TLS.

Connectivity over the internet

Amazon MSK offers an option to securely connect to the brokers of Amazon MSK clusters running Apache Kafka 2.6.0 or later versions over the internet. By enabling Public Access, authorized clients external to a private Amazon Virtual Private Cloud (VPC) can stream encrypted data in and out of specific Amazon MSK clusters.

Deeply integrated

No other provider offers the breadth and depth of AWS integrations in Amazon MSK. These integrations include:

  • AWS IAM for Apache Kafka and service-level API access control
  • Amazon Kinesis Data Analytics for running fully managed Apache Flink applications to process streaming data within Apache Kafka
  • Amazon Kinesis Data Analytics Studio to run interactive Streaming SQL and long-running SQL jobs using Apache FlinkSQL
  • AWS Glue Schema Registry to centrally control and evolve schemas
  • AWS IoT Core for IoT event streaming into MSK
  • AWS Database Migration Service (AWS DMS) for change data capture and analytics
  • Amazon Virtual Private Cloud (Amazon VPC) for private client connectivity and network isolation
  • AWS Key Management Service (AWS KMS) for at-rest encryption
  • AWS Certificate Manager Private Certificate Authority for mutual TLS client authentication
  • AWS Secrets Manager for secure storage and management of SASL/SCRAM secrets
  • AWS CloudFormation to deploy Amazon MSK in code
  • Amazon CloudWatch for cluster-, broker-, topic-, consumer-, and partition-level metrics

Open Source

Run with native Apache Kafka

Amazon MSK deploys native versions of Apache Kafka so applications and tools built for Apache Kafka just work with Amazon MSK out of the box, with no application code changes.

Streamlined version availability

Amazon MSK typically makes newer versions of Apache Kafka available within seven days of public availability.

Seamless version upgrades

You can upgrade Apache Kafka versions on provisioned clusters in just a few clicks, allowing you to decide when to take advantage of features and bug fixes present in new Apache Kafka versions. Amazon MSK automates the deployment of version upgrades on running clusters to maintain client I/O availability for customers following best practices. For serverless clusters, Apache Kafka versions are upgraded automatically by Amazon MSK.

Lowest cost

Amazon MSK lets you get started for less than $2.50 per day. Customers typically pay between $0.05 and $0.07 per GB ingested, all-in, which can be as low as 1/13th the cost of other managed providers. Visit the Amazon MSK Pricing page to learn more about pricing.

Scalable

Broker scaling (provisioned clusters only)

You can scale your Amazon MSK clusters by changing the size or family of your Apache Kafka brokers in minutes with no downtime. Changing the size or family is a popular way to scale Amazon MSK clusters because it gives you the flexibility to adjust cluster compute capacity for changes in your workloads. This method can be preferred because it doesn’t require partition reassignment, which can impact Apache Kafka availability.

Cluster scaling (serverless clusters only)

Amazon MSK automatically scales compute and storage resources of your clusters in response to your application’s throughput needs.

Automatic partition management

Amazon MSK integrates with Cruise Control, a popular open-source tool for Apache Kafka that automatically manages partition assignment on your behalf. For serverless clusters, Amazon MSK automatically manages partition assignments for you.

Automatic storage scaling (provisioned clusters only)

You can seamlessly scale up the amount of storage provisioned per broker to match storage requirement changes using the AWS Management Console or AWS Command Line Interface (AWS CLI). You can also create an auto scaling policy to automatically expand your storage to meet growing streaming requirements.

Configurable

Amazon MSK deploys a best practice cluster configuration for Apache Kafka by default. For provisioned clusters, you have the ability to tune more than 30 different cluster configurations while supporting all dynamic and topic-level configurations. For more information, see Custom MSK Configurations in the documentation.

Visible

Easy observability of streaming performance with CloudWatch metrics by default

You can visualize and monitor important metrics using Amazon CloudWatch to understand and maintain streaming application performance.

Export JMX and Node metrics to a Prometheus server with Open Monitoring (provisioned clusters only)

Open Monitoring with Prometheus lets you monitor Amazon MSK using solutions such as Datadog, Lenses, New Relic, Sumo Logic, or a Prometheus server, and easily migrate your existing monitoring dashboards to Amazon MSK. For more information, see Open Monitoring with Prometheus in the documentation.

Get started with Amazon MSK

Calculate your costs
Calculate your costs

Visit the Amazon MSK pricing page.

Read the documentation
Review the Get Started guide

Learn how to set up your Apache Kafka cluster on Amazon MSK in this step-by-step guide.

Run your Apache Kafka cluster
Run your Apache Kafka cluster

Start running your Apache Kafka cluster on Amazon MSK. Log in to the Amazon MSK console.