Real-time streaming data top picks you cannot miss at AWS re:Invent 2023

Save the date: AWS re:Invent 2023 is happening from November 27 to December 1 in Las Vegas, and you cannot miss it. re:Invent is a learning conference organized by AWS for the global cloud computing community. It’s a great opportunity to come together with cloud enthusiasts from around the world to hear about the latest cloud industry innovations, meet with AWS experts, have fun, and build connections.

Join us as we delve into the world of real-time streaming data at re:Invent 2023 and discover how you can use real-time streaming data to build new use cases, optimize existing projects and processes, and reimagine what’s possible. In today’s data-driven landscape, the quality of data is the foundation upon which the success of organizations and innovations stands. High-quality data is not just about accuracy; it’s also about timeliness. To derive meaningful insights and ensure the optimal performance of machine learning (ML) and generative AI models, data needs to be ingested and processed in real time. Real-time data empowers these models to adapt and respond instantaneously to changing scenarios, making them not just smarter but also more practical. With real-time streaming data, organizations can reimagine what’s possible. From enabling predictive maintenance in manufacturing to delivering hyper-personalized content in the media and entertainment industry, and from real-time fraud detection in finance to precision agriculture in farming, the potential applications are vast.

This post will help you plan your re:Invent experience by highlighting the essential sessions on real-time streaming data and beyond that you shouldn’t miss. To attend these sessions, be sure to register for re:Invent and access the session catalog. Register now!

Keynotes and Innovation Talk sessions you cannot miss!

Don’t miss your chance to hear from some of the leading voices in AWS. Here are some of our favorite keynotes and Innovation Talk sessions.

Adam Selipsky, Chief Executive Officer of Amazon Web Services – Keynote

Tuesday, November 28 | 8:30 AM – 10:30 AM PST | The Venetian

Join Adam Selipsky, CEO of Amazon Web Services, as he shares his perspective on cloud transformation. He highlights innovations in data, infrastructure, and artificial intelligence and machine learning that are helping AWS customers achieve their goals faster, mine untapped potential, and create a better future.

Reserve your seat now!

Swami Sivasubramanian, Vice President of AWS Data and AI– Keynote

Wednesday, November 29 | 8:30 AM – 10:30 AM PST | The Venetian

Join Swami Sivasubramanian, Vice President of Data and AI at AWS, to discover how you can use your company’s data to build differentiated generative AI applications and accelerate productivity for employees across your organization. Also hear from customer speakers with real-world examples of how they’ve used their data to support their generative AI use cases and create new experiences for their customers.

Reserve your seat now!

Putting your data to work with generative AI – Innovation Talk

Thursday, November 30 | 12:30 – 1:30 PM PST | The Venetian

Join Mai-Lan Tomsen Bukovec, Vice President, Technology at AWS to learn how you can turn your data lake into a business advantage with generative AI. Explore strategies for putting your proprietary datasets to work when building unique, differentiated generative AI solutions. Learn how to utilize your datasets using Amazon SageMaker and Amazon Bedrock and popular frameworks like PyTorch with AWS compute, storage, and analytics. Hear best practices for using unstructured (video, image, PDF), semi-structured (Parquet), and table-formatted (Iceberg) data for training, fine-tuning, checkpointing, and prompt engineering. Also hear different architectural patterns that customers use today to harness their business data for customized generative AI solutions.

Reserve your seat now!

Data drives transformation: Data foundations with AWS analytics – Innovation Talk

Thursday, November 30 | 2:00 – 3:00 PM PST | The Venetian

Join G2 Krishnamoorthy, Vice President of AWS Analytics, to discuss strategies for embedding analytics into your applications and ideas for building a data foundation that supports your business initiatives. With new capabilities for self-service and simpler builder experiences, you can democratize data access for line-of-business users, analysts, scientists, and engineers. Hear inspiring stories from adidas, GlobalFoundries, and the University of California, Irvine.

Reserve your seat now!

Breakout sessions

Deepen your learning with re:Invent breakout sessions. re:Invent breakout sessions are lecture-style and 1 hour long. These sessions take place across the re:Invent campus and cover all topics at all levels. The following are some of our favorites:

• Accelerate innovation with real-time data: Join Mindy Ferguson, Vice President of Streaming and Messaging at AWS, and Arvinth Ravi, General Manager of Amazon Kinesis Data Streams, in this session that highlights the importance of implementing ubiquitous real-time data strategies to gain a competitive edge. Discover the latest launches in AWS streaming data services, gain insights into real-world applications, and explore how you can use them to solve a variety of use cases to make quick, real-time decisions to optimize costs, increase customer engagement, and drive growth. Reserve your seat now!
• How Rivian builds real-time analytics from electric vehicles: Learn how Rivian’s vehicle data platform team uses Amazon MSK and other AWS analytics services to build a secure, scalable, cost-effective, and extensible data platform that powers different teams across the organization. Reserve your seat now!
• Serverless data streaming: Amazon Kinesis Data Streams and AWS Lambda: Explore the intricacies of creating scalable, production-ready data streaming architectures using Amazon Kinesis Data Streams and AWS Lambda. Delve into tips and best practices essential to navigating the challenges and pitfalls inherent to distributed systems that arise along the way, and observe how AWS services work and interact. Reserve your seat now!
• Break down data silos using real-time synchronization with Flink CDC: Learn how, by using Flink CDC on AWS, you can simplify real-time synchronization and ingestion across transactional databases and finally break down data silos. Reserve your seat now!
• Scaling serverless data processing with Amazon Kinesis and Apache Kafka: Explore how to build scalable data processing applications using AWS Lambda. Learn practical insights into integrating Lambda with Amazon Kinesis and Apache Kafka using their event-driven models for real-time data streaming and processing. Design serverless data processing pipelines and extract valuable insights from real-time data streams. Reserve your seat now!

Chalk talks

Chalk talks are a highly interactive content format with a small audience. Each begins with a short lecture delivered by an AWS expert followed by a Q&A session with the audience.

• Stream data and build transactional lakes with AWS Glue streaming: Learn how to create a low-code streaming pipeline with AWS Glue and Amazon Kinesis to load, clean, and transform sales data, and make it available for machine learning. Reserve your seat now!
• Migrating self-managed Apache Flink to fully managed on AWS: Review key details to keep in mind when migrating self-managed Apache Flink applications to Amazon Managed Service for Apache Flink. Explore how using cloud-based services with the right patterns can reduce operational overhead. Hear about best practices that can help with migration while still maintaining costs, scale, and performance. Reserve your seat now!
• How to migrate your Apache Kafka workloads to Amazon MSK: Amazon Managed Streaming for Apache Kafka (Amazon MSK) is an AWS streaming data service that manages Apache Kafka infrastructure and operations. Learn the benefits of migrating to Amazon MSK and how to lift and shift existing clusters using MirrorMaker 2.0 (MM2). Take a deep dive into the MM2 and Kafka Connect architecture, and explore the common bottlenecks and challenges of high-throughput data streams. Reserve your seat now!
• Accelerating value from data: Migrate from batch to stream processing: Explore the competitive advantages of transitioning from traditional batch processing to stream processing. Look at key concepts and architectural patterns of streaming to address potential challenges and hiccups along this migration journey. Through an example use case, gain insights into how to successfully transition from batch to real-time data processing using AWS streaming data services. Reserve your seat now!
• Fast-track streaming ETL with AWS streaming data services: Learn how to build streaming data pipelines across data lakes and data warehouses. Learn best practices for performance, scale, and cost control in Amazon Kinesis Data Streams, Amazon MSK, Amazon Redshift streaming ingestion, and AWS Glue streaming. Reserve your seat now!
• Advanced real-time analytics and ML in your data warehouse: Learn how enterprise customers are rearchitecting their data environments to lean more heavily on streaming analytics to process throughput of millions of records per second. We will show you how to implement an ELT approach that uses Amazon Redshift’s native integration with streaming data services and Amazon Redshift ML to build near-real-time ML predictions in SQL for better decision-making. Reserve your seat now!
• Building better with Apache Kafka and Amazon EventBridge: Explore the convergence of event-driven architectures and real-time event streaming to meet modern business requirements. Learn about Amazon EventBridge and Apache Kafka and gain insights into their roles for facilitating real-time decision-making, increased business velocity, and operational autonomy. Reserve your seat now!
• Speed up data predictions with real-time inference at scale: It is crucial to analyze, learn, and make predictions from your data in a timely manner because the value of data diminishes over time. Dive into event-driven architecture and see how you can use machine learning inference with streaming data to detect patterns and anomalies to realize a faster path to decisions from your data. Reserve your seat now!

Code Talk

Code Talks are 60-minute sessions that focus on the actual code involved in building a solution. You’ll learn the “why” behind the solution and see it come to life—complete with the inevitable errors. Your questions are welcome and encouraged.

• Build a Flink application on Amazon Managed Service for Apache Flink: Join this code talk to develop and host a streaming and production-ready Apache Flink application on Amazon Managed Service for Apache Flink using the DataStream and Table API programming models. Reserve your seat now!

Workshops

Workshops are 2-hour hands-on sessions where you work in teams to solve problems using AWS services. Workshops organize attendees into small groups and provide scenarios to encourage interaction, giving you the opportunity to learn from and teach each other. Don’t forget to bring your laptop!

• Event detection with MSK and Amazon Managed Service for Apache Flink: Take on the role of a technology manager for a Las Vegas casino. Learn how to create a stream-processing application that identifies historically “high roller” casino guests entering the casino and that sends you an email when they sit down at a gambling table. Reserve your seat now!

Bootcamp

Bolster your confidence with AWS services and solutions in these eight-hour, deep-dive sessions. Learn by doing with expert guidance, immersive exercises, and self-paced labs.

The below is one you cannot miss:

• Building streaming data analytics solutions on AWS: Learn to design and implement streaming data analytics solutions using AWS services with an expert AWS instructor. This 8-hour, deep-dive bootcamp is designed for data engineers, solutions architects, and developers who want to build and manage streaming data analytics pipelines. This bootcamp covers how to scale streaming applications using Amazon Kinesis; how to optimize data storage; how to select and deploy appropriate options to ingest, transform, store, and analyze data; and more. You can practice new skills in a real AWS environment with hands-on labs that cover data analytics solutions. Reserve your seat now!

Still time for FUN!

All work and no play … not at re:Invent! Sure, we’ll work hard and learn a lot, but we also plan to have a great time while we’re together. AWS Builder Labs will give you the opportunity to test your skills in sandbox settings while working alongside some of the leading minds from AWS!

Join the PeerTalk platform to easily find, connect, and network with others at re:Invent. Opt in to discover attendees who share similar interests—PeerTalk recommends connections for you. Browse recommendations, message people, and arrange meetups through the platform, then sit down for a face-to-face conversation in one of the four PeerTalk lounges located throughout the re:Invent campus.

Stop by the Analytics kiosk in the re:Invent expo hall. Connect with experts to dive deeper into AWS streaming data services such as Amazon Kinesis Data Streams, Amazon Kinesis Data Firehose, Amazon Managed Service for Apache Flink, and Amazon Managed Streaming For Apache Kafka (Amazon MSK). Also, win prizes, show off your Analytics superhero, and learn all about the latest innovations from our AWS Analytics services.

Register today

It’s going to be an amazing event, and we can’t wait to see you at re:Invent 2023! Register now to secure your spot!

About the author

Anna Montalat is a Senior Product Marketing Manager for AWS streaming data services which includes Amazon Managed Streaming for Apache Kafka (MSK), Kinesis Data Streams, Kinesis Video Streams, Kinesis Data Firehose, and Kinesis Data Analytics. She is passionate about bringing new and emerging technologies to market, working closely with service teams and enterprise customers. Outside of work, Anna skis through winter time and sails through summer.