AWS Open Source Blog
Author: Patrick Ames
Patrick Ames is a Principal Engineer working on data management and optimization for big data technologies at Amazon.
Amazon’s Exabyte-Scale Migration from Apache Spark to Ray on Amazon EC2
Large-scale, distributed compute framework migrations are not for the faint of heart. There are backwards-compatibility constraints to maintain, performance expectations to meet, scalability limits to overcome, and the omnipresent risk of introducing breaking changes to production. This all becomes especially troubling if you happen to be migrating away from something that successfully processes exabytes of […]