The Two-Week Rule: Refactor Your Applications for the Cloud in 10 Days

I promise this is not clickbait. In my last post, The Great Cloud Refactoring Debate, I introduced the Two-Week Rule as one example for enterprises planning a migration to the cloud. The Two-Week Rule is not a theoretical approach, but an actual cloud migration mechanism that we developed during my time at Edmunds.com to complete on schedule a two-year migration of the company’s entire web-scale infrastructure to AWS. This ten-day, five-step process was a beacon of consistency and purpose for our teams during periods of uncertainty and risk while adopting a new infrastructure platform.

Joining me for this post is my former Edmunds.com colleague Ajit Zadgaonkar. Ajit was the Executive Director of Engineering at Edmunds.com and also led the cloud engineering and migration teams that created the Two-Week Rule.

Edmunds.com was like many enterprises starting out with the cloud: we were not sure our current applications would run in a cloud environment without significant changes. But exactly how much change was needed? The team did not start with a detailed five-step process; instead they began by migrating several applications with a simple overarching constraint: stand up the application in a target AWS cloud environment within ten days. This constraint did not specify a subjective level of refactoring, but by getting started and limiting work to ten days per application it focused the effort on learning the minimum amount of change needed to move an application to the cloud. As more applications were moved with nominal changes, repeatable patterns were identified that could be automated to accelerate the overall migration. The migration teams began taking on multiple application migrations in parallel, using increasingly standardized move templates and, at times, migrating tens of applications during a two-week sprint.

Eventually, the Two-Week Rule was solidified into a five-step process that was continuously optimized throughout the migration to the point where tasks were detailed in a daily schedule over a ten-day period. In effect, we were giving the application development teams at Edmunds.com an opportunity to work hand-in-hand with the cloud migration teams on refactoring the most problematic compatibility issues they felt their applications might have running in the cloud.

Ajit, who has since joined AWS to lead our DevOps practice in AWS Professional Services, has provided a summary of each step below that, regardless of the number of days allotted, provides a proven framework for how to manage and limit the steps required to move applications to the cloud.

The Two-Week Rule: Step-by-Step and Day-by-Day

Step 1 of 5: Days 1 and 2 (Review and Update Plan)
Getting a running start is important, and like many enterprises we had already built a migration project plan that aligned applications into a sequential order of migration, or move groups. Leveraging our Application Performance Monitoring (APM) solution, Chef cookbooks, and other system runbooks we also had a good idea of the dependency considerations, so we focused the first two days on validating this information for completeness and accuracy. Also, since we were redeploying these applications on AWS, we ensured that the existing monitoring, health checks, unit/integration, and load tests were going to be a sufficient validation of the new cloud-based environment. This might go without saying, but unless you have already built, secured, and operationalized your target cloud environments, you won’t be able to progress through this step in two days for any application. There are many paths enterprises can follow to achieve this, from following the AWS Cloud Adoption Framework (CAF), to using an AWS partner-run migration program, or leveraging the new AWS Landing Zone solution to build out a secure target environment.

Step 2 of 5: Days 3 and 4 (Cleanup)
Early on in the migration we observed that our review process was consistently identifying custom runtime properties and override configurations that would cause problems when migrating an application to the cloud. The second step became a clean up effort, and over time the migration teams were able to automate the discovery and remediation of these landmines. Teams also used this step for limited refactoring that usually centered around dropping old dependencies and libraries no longer being actively utilized by the app, especially ones that were being replaced by cloud native services like logging frameworks. Depending on how well the application development teams kept their platform current, this was an opportunistic window to update the platform versions (Java, Node.js, Tomcat,  and libraries—logging/Spring).

Step 3 of 5: Days 5 and 6 (Cloud-a-Mize Applications)
Working hand-in-hand with the cloud engineering/migration teams, the application development team that owned the application(s) being migrated typically spent two full days refactoring the environment/infrastructure specific configurations for the cloud. Ops engineering teams would write AWS CloudFormation templates. Chef cookbooks are being updated to specify the deployment of AWS cloud infrastructure instead of using our on-premises data center infrastructure. Application development teams would spend these two days finishing the update of platform changes, building new APM configurations, and completing other changes to make their application runtime properties compatible with the new cloud environment. Storage architecture was perhaps the biggest refactoring effort we needed to undertake for virtually every application we migrated, so ensuring systems were configured to properly leverage Amazon S3 object storage or Amazon Elastic Block Store (EBS) was a critical component of this step. The two goals for this step were to have AWS specific Chef cookbooks and a successful application build on our CI server with artifacts ready to be deployed.

Step 4 of 5: Day 7 (Deploy and Observe)
The day of truth: deploying the refactored application into an AWS environment (Amazon VPC). With a few exceptions, the teams were successful in getting hundreds of applications up and running during this one-day allocation of time. Once repeatability of deployment was validated, all teams involved in the sprint observed the stability and performance of the application(s) like a new parent listens to a baby monitor. Tweaks were made if needed to proceed to the next step.

Step 5 of 5: Days 8, 9, 10 (Bake, Burn, Crash, Restart, and Fine Tune for Cloud)
The tasks during the final three days of the sprint were concerned with testing and re-baselining applications. Getting functional tests to pass was key because the team often discovered URLs, ports, and contexts that were hardcoded, both in the tests as well as in the applications. The next action was to load test applications with their new AWS cloud configuration, eventually ramping up to 200% of the expected load. This stress testing was critical to creating a new performance and resiliency baseline for each application, understanding how new cloud characteristics like auto-scaling performed or understanding the impact of one or more application node failures/restarts. Final sign-off included updated runbooks, security/data protection review, and new monitors for cloud resources.

Not every application fit into the ten-day window or five-step process, but those were the few exceptions, and they still followed the same overall workflow outlined above. There are other approaches from customers, consulting partners, and ones developed by AWS that can have similar outcomes, and we will continue to share those that demonstrate consistent results. Some are generic methodologies with broad application, while others are highly targeted to specific application or database types. Fifty apps in fifty days is one such approach developed with AWS Managed Services to kick start an AWS migration and build momentum quickly within an enterprise. I like to think of the Two-Week Rule as a mechanism to sustain a long-term migration with a consistent cadence when hundreds or thousands of applications need to be moved over a multi-year period.

Do you love constraints, or maximizing the amount of work NOT done to achieve goals? Ajit and I are looking for more discussion and debate on lean cloud adoption strategies that speed up time to value in the cloud. Please get in touch to share your approach or stay tuned to continue to learn how others are doing it.

while(life) {refactor}

Philip Potloff
Head of Enterprise Strategy, AWS
@philippotloff
LinkedIn

Ajit Zadgaonkar
Leader – Global DevOps Specialty Practice, AWS
LinkedIn