Customer Stories / Financial Services / United States

FactSet Logo

Improving Developer Deployment Times from Weeks to Minutes Using Amazon MSK with FactSet

Learn how FactSet reduced costs by 56 percent while improving availability and productivity using Amazon Managed Streaming for Apache Kafka.


cost savings


high availability


deployment times from weeks to minutes




FactSet Research Systems Inc. (FactSet), which provides data and software solutions to investment professionals worldwide, wanted to migrate its on-premises Kafka clusters to the cloud so its logging solution could be fully cloud based. With its old solution, FactSet’s dedicated infrastructure team needed to spend time addressing common pain points associated with legacy Kafka clusters, such as maintenance and patching. Since the on-premises cluster was not the team’s primary responsibility, this led to a longer wait time for requests and decreased productivity. Besides aligning with the company’s hybrid cloud strategy, FactSet wanted to use this migration to implement an overall strategy of improving DevOps productivity—for example, by fulfilling logging requests for internal teams more quickly.

The company had already been using Amazon Web Services (AWS) for most of its infrastructure, so it was familiar with the benefits. To migrate its Kafka clusters to the cloud, FactSet chose Amazon Managed Streaming for Apache Kafka (Amazon MSK), a fully managed service that facilitates efficient ingestion and processing of streaming data in near real time. By migrating its clusters to Amazon MSK, FactSet has increased productivity, achieved high availability, and reduced operational costs by 56 percent.

Developing programmer Team Development Website design and coding technologies working in software company office

Opportunity | Using Amazon MSK to Increase Productivity for FactSet

FactSet provides access to financial data and analytics through its software solutions to help its clients make sound and efficient investing decisions. Founded in 1978, the company delivers financial data, analytics, and open technology to over 7,500 global institutions, including more than 185,000 individual users. Although the company had already been using AWS for other parts of its stack, it had continued to manage its Kafka clusters on premises. FactSet’s observability team, whose primary responsibilities included providing logging for internal stakeholders such as business analytics teams and site reliability engineering teams, wanted to make its Kafka cluster management more time and cost effective. By migrating its Kafka clusters to the cloud, the observability team could reduce wait times and make the experience more seamless for its stakeholders. “We wanted a solution where we didn’t have to manage the infrastructure and instances directly,” says Chris Bertsch, principal software engineer at FactSet. “Finding a fully managed solution for Kafka was an ideal situation for us.”

The Amazon MSK migration took about 6 months, and the company achieved this all in-house without needing third-party support. The team experienced virtually zero downtime during the migration, using Apache Kafka mirroring (MirrorMaker) to mirror the traffic from the on-premises Kafka cluster to Amazon MSK. After a period of running in shadow mode while replicating in the cloud, the actual migration cutover took a mere week. “With numerous internal stakeholders connecting to our Kafka cluster, it was integral that we coordinated a time frame with them,” says Bertsch. “We were happy with the migration time frame.”


The tiered storage feature of Amazon MSK has been a game changer for us. It has greatly reduced our costs.”

Chris Bertsch
Principal Software Engineer, FactSet

Solution | Reducing Costs by 56 Percent Using Tiered Storage in Amazon MSK

Migrating to Amazon MSK has saved time for FactSet’s observability and other internal teams, who had been tediously coordinating among themselves to configure their on-premises Kafka clusters. Now, with full ownership of the stack, the observability team can scale its compute resources at will without having to submit tickets to the infrastructure team. This scalability improves productivity and provides the team with the ability to support an overall DevOps strategy to accelerate the process of providing features, applications, and logs. Similarly, the infrastructure team, which formerly spent its time managing the technological aspects of the on-premises Kafka clusters, now has the bandwidth to focus on other projects. As a result, the company’s stakeholders have felt the benefit as well. “Previously when we had to start sending more logs to different consumers, we would have to wait for 2 to 3 weeks,” says Bertsch. “Now, since we fully manage virtually all aspects through Amazon MSK, we can make those changes in a matter of minutes.”

After migrating its clusters to Amazon MSK, FactSet reduced its operational costs due to the tiered storage feature within Amazon MSK. Previously, with the on-premises Kafka cluster, the company stored hot data, which demands swift retrieval, for 7 days in case backfill and reprocessing were necessary. Using the various tiers of cloud storage in Amazon MSK, the company has gained greater flexibility to differentiate between hot data and more rarely accessed cold data. Hot data is stored in performance-optimized primary storage, whereas cold data is automatically migrated into the low-cost storage tier. In addition, the tiered storage feature is fully managed and does not require provisioning, which means it can scale to virtually unlimited storage. “The tiered storage feature of Amazon MSK has been a game changer for us,” says Bertsch. “It has greatly reduced our costs.” Overall, the company has used this functionality to reduce its costs by 56 percent.

The company has also improved overall availability, which has been crucial to serving its international clientele. Whereas the previous legacy system was located in a single data center, migrating the clusters to the cloud has granted the team the option to deploy its services across different AWS Availability Zones. Furthermore, using Amazon MSK, the company has improved the elasticity of its infrastructure and can scale up in a pinch. “By being on AWS, we can burst and scale at ease as the marketplace changes,” says Bertsch. “It was vital for us to get all of our infrastructure on the cloud, including our Kafka elements.”

Outcome | Expanding Across Multiple Regions Using AWS

Looking ahead, FactSet plans to use the increased availability of its infrastructure across multiple AWS Availability Zones to find growth opportunities for the company.

“The next big thing for us is to be multiregion,” says Bertsch. “Previously, all our Kafka infrastructure was in a single data center, and the latencies between different regions were far too high. But now, the feasibility of deploying Amazon MSK to virtually any region opens up brand-new possibilities.”

About FactSet Research Systems Inc.

FactSet Research Systems Inc. delivers financial data, analytics, and open technology to over 185,000 global financial professionals across the buy-side and sell-side, as well as wealth managers, private equity firms, and corporations.

AWS Services Used

Amazon Managed Streaming for Apache Kafka (Amazon MSK)

Amazon MSK makes it easy to ingest and process streaming data in real time with fully managed Apache Kafka.

Learn more »

AWS Availability Zones

The AWS Cloud spans 102 Availability Zones within 32 geographic regions around the world, with announced plans for 12 more Availability Zones and 4 more AWS Regions in Canada, Malaysia, New Zealand, and Thailand.

Learn more »

More Financial Services Customer Stories

no items found 


Get Started

Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.