Lenses Stream Reactor: S3 Source & Sink Kafka Connectors

An Apache Kafka Connect plugin for Amazon S3. As both a source and sink, it holds powerful enterprise features to support many data integration use cases including backup/restore of Kafka topics, data replication across Kafka clusters, sourcing multi-line events/XML and hydrating a data warehouse with optimized object partitioning. It works on any Kafka Connect cluster including AWS MSK Connect. The plugin is licensed under Apache 2.0, open source and available with optional enterprise support via a subscription with Lenses.io.

Request private offer

Overview

An enterprise-grade Apache Kafka Connect Source & Sink plugin for Amazon S3 designed to support critical business flows.

The connectors hold powerful enterprise features.

For sourcing: Collecting unstructured data including multi-line events & XML Task parallelization for optimal throughput Dynamic S3 partition discovery

For sinking: Support for writing to AVRO, Parquet, JSON, Bytes and Text Full object export (key, value, metadata and headers) Object partitioning by key, value, headers and time window Exactly-once semantics Hive-style (eg. key=value) object partitioning

This widely used connector integrates Kafka and S3 to support a number of use cases including hydrating a lakehouse, cost-efficient Kafka topic backup/restore, and data replication to another Kafka cluster.

It can be deployed as a standalone plugin for any Apache Kafka Connect cluster (including MSK Connect) or deployed via Lenses for a simplified single click management with role-based security, auditing, monitoring, and governance.

The plugin is licensed under Apache 2.0, open-source and available with enterprise support from Lenses.io.

Contact info@lenses.io for details about enterprise support offerings.

Highlights

- Supports a number of S3 data integration use cases including backup/restore of Kafka topics, data replication to active Kafka clusters, sourcing complex data formats including XML & unstructured data from S3 - Offers optimized partitioning to sink into AWS Glue, Redshift and Athena - Fully open-source under Apache 2 license - Enterprise-grade, developed in partnership with a number of global organizations.
- Developed by the Lenses.io team, the biggest contributors to Apache 2 connectors in the Kafka community - Available with enterprise support via Lenses.io - Works with self-managed Apache Kafka Connect & MSK Connect - Enhanced Developer Experience when managed through Lenses.io

Details

Sold by

Lenses

Pricing

Custom pricing options

Request private offer

Pricing is based on your specific requirements and eligibility. To get a custom quote for your needs, request a private offer.

Legal

Content disclaimer

Vendors are responsible for their product descriptions and other product content. AWS does not warrant that vendors' product descriptions or other product content are accurate, complete, reliable, current, or error-free.

Support

Vendor support

The Connector is open source under an Apache 2.0 license.

Full enterprise support is offered for subscribed Lenses.io customers (see SLAs ). See full product description . Contact info@lenses.io for information on enterprise support offerings.

Engage with the Lenses community via the Community Slack or ask technical questions via Ask Marios .