Stream computing and event driven architectures deliver a modern, cloud-native design pattern to process large amounts of data in near real-time. The learning curve to adopt these technologies remains steep, especially for organizations with a more traditional three tier architecture for analytics applications.

To help large enterprises overcome initial hesitance and shorten POC durations, Ness has developed a quick start package specifically for capital markets firms seeking to explore stream computing. The ready to deploy package contains Infrastructure as Code (IaC) to deploy a stream computing platform and a scalable application which performs analytics on a test data set.

This quick start package, which we call Steel Thread, stands up Amazon Managed Streaming for Apache Kafka (MSK) and Managed Service for Apache Flink (MSF) clusters. MSK serves as the event streaming store, while MSF is the stream computing platform. We provide scripts to inject test data into the MSK cluster, and a demo application is deployed onto the MSK cluster. Instructions are provided so your engineers can experiment with parallelism on the MSF cluster and observe performance characteristics of Kafka and Flink.

Configuring and deploying the platform and application into a new AWS account takes less than half a day, and you can have a demo-ready environment by mid-day. This reference implementation can also be deployed into a sandbox for your developers to begin learning how to develop and deploy streaming computing applications.