DevOps and Cloud Migrations – A Perspective from AWS DevOps and Migration Competency Partner Pythian

The following is a guest post from our friends at AWS DevOps & AWS Migration Competency Partner Pythian. 

It’s no secret that the commoditization of computing resources, and cloud in particular, have removed a significant barrier to entry for organizations seeking platform scalability, elasticity, and rapid maturation. Cloud computing has created an opening for organizations of all sizes to compete on level ground, while aligning their costs of operations with demand and effectively supporting a more sustainable financial model. The broad service offering and the presence of reliable DevOps tools offered by AWS, such as AWS CloudFormation , AWS OpsWorks, AWS CodeDeploy, AWS CodePipeline, AWS CodeCommit, AWS Elastic Beanstalk, and AWS Config support automated, DevOps-style migrations of platforms of all sizes and complexities.

Migrating a platform to the cloud can be tackled in numerous different ways. Whether the migration is approached via the refactoring strategy or down to the traditional re-host strategy, the primary driver is to enable speed, agility and the freedom to experiment in a cost-efficient manner for your organization. Regardless of which route is chosen, they all require strategic planning and precise execution. If not planned well or if done incorrectly, a migration could result in downtime or potentially not allowing your organization to take full advantage of what the cloud has to offer.

Migrations require careful, granular planning, and a deep, cross-discipline understanding of platform intricacies. They also require the involvement and cooperation of a wide range of domain experts — from platform architects to operations experts, to developer resources — and a high degree of orchestration and focus on maintaining team coherence.

To resolve these incompatibilities, bridge the various disciplines, and realign the previously separate interests, advanced technology teams have been evolving their DevOps practices.

AWS provides deep guidance on what DevOps is, why it matters, and how to adopt a DevOps model. AWS defines DevOps as the combination of cultural philosophies, practices, and tools that increases an organization’s ability to deliver applications and services at high velocity: evolving and improving products at a faster pace than organizations using traditional software development and infrastructure management processes. This speed enables organizations to better serve their customers and compete more effectively in the market. Adopting a DevOps model can have a fundamental impact on the success of your migration to the cloud.

DevOps supports an experiment-driven, fast-learning, flexible, and high-velocity engineering culture. It is designed to automate the repeatable parts and sequences of an application’s, and its supporting infrastructure’s, lifecycle (like build, test, deploy, scale, failover, recover, etc.) to make changes to the resulting software and production services as close to zero-cost as possible. The core DevOps principles of shared responsibility, agility, transparency, and lowered risks through heavy reliance on automation have proven to be crucial for handling complex projects requiring constant and precise risk management, such as enterprise platform migrations.

Early adoption and a commitment to DevOps practices enables low-risk execution of AWS migrations. DevOps also brings about a number of short and long-term business benefits, such as repeatability, auditability, significantly lower deployment risks, and faster iteration cycles – all crucial to employee retention, improved operational excellence, better security posture, and improved competitive advantage.

DevOps teams that have migrated their infrastructure and workload to the cloud using automation, have now enabled their organizations to run more experiments at significantly lower cost than previously possible, and to iterate at a more rapid pace than ever before.

Today, the most advanced teams in the world release each developer’s changes automatically to huge environments many times per day. Deploying more often and in smaller increments allows for significant risk reduction at the time of deployment. These smaller change deltas, coupled with fast feedback loops enabled by rigorous automated testing of each change increment, reduce the time it takes to find and correct bugs and further reduces deployment risks.

The Road to Maturity: Elements of a Mature DevOps Practice

We’ve learned a lot helping numerous customers migrate to AWS and build mature DevOps practices within their organization, and we want to share some best practices with you. Below is a brief overview of the environment we suggest you create, and the technologies your organization should embrace, to ensure ongoing improvements to your software and services to achieve your business goals.

CULTURAL SHIFT

• Create a culture of experimentation: Every product organization must make ongoing tradeoffs between velocity, performance, reliability, cost efficiency, and security, so it’s imperative to have blameless, respectful, and truthful discussions about decisions and outcomes. Adopting DevOps requires a deep commitment to continuous improvement throughout the organization. Continuous experimentation with a deep commitment to constant learning leads to continuous improvement, also known as “Kaizen”. The successful organization will learn to seek answers and limit the risk of bad assumptions. AWS Head of Enterprise Stephen Orban has also released a great series of blogs on creating a culture of experimentation, which you can read here.
• Get executive attention: The modern executive team should concern itself with improving its engineering leverage, and must be willing to seek help through the inherent disruption.
• Be customer focused: Commercial innovation is judged by the market. Empathy for the customer is a major differentiator.
• Embrace tool-driven collaboration: Mature DevOps teams are characterized by deep, efficient, tool-driven collaboration between skilled, cross-functional individuals working as part of a team, or teams, who execute a product development and operations lifecycle.
• Adopt data-driven decision making: Success comes from making good decisions over time. Business strategy and financial planning cycles must adapt to the reality of continually evolving markets by constantly seeking alignment with product and technology objectives — and vice versa. This requires an even closer, data-driven decision cycle at all levels.
• Get Lean and Agile: Adopting Lean principles and Agile methodologies is crucial to delivering DevOps in a highly productive, scalable, and efficient manner.

TECHNICAL PRACTICES

• Take an infrastructure-as-code approach: All development efforts should be cloud native, incorporate an infrastructure-as-code approach using tools like AWS CloudFormation, and avoid dependence on manual operations by leveraging appropriate public, private and/or hybrid cloud infrastructure. This approach allows you to automate manual and repetitive tasks and eliminate human errors as you configure, maintain, scale, and recover from faults, and ultimately fully automate the end-to-end infrastructure lifecycle.
• Automate application deployments: Application deployments should be guided by the principle of delivering value to the end user as quickly as possible. To attain high velocity, deployments must be automated, undergo rigorous automated testing, and upon successfully passing the tests, deployed to production. The entire software deployment pipeline should be optimized for defect detection, speed of delivery, and rapid feedback, with Continuous Integration (CI) and Continuous Delivery (CD) as the goal.
• Optimize orchestration: Design, coordinate, and optimize the full infrastructure and application lifecycle processes by creating fast feedback loops, eliminating repetitive steps to optimize the speed and quality of new application capabilities.
• Integrate operational visibility early: Integrate operational visibility services as part of the development process, not as an afterthought.  Select and configure tools to monitor KPIs to maintain performance, detect issues, and provide alerts when those issues arise. Use pre-built operational visibility platforms like Pythian’s OpsViz stack, Amazon CloudWatch, Datadog, Splunk, Loggly, or Sumo Logic for general data aggregation, dashboards, triggering of actionable alerting, as well as reporting. Drive development efficiency and service quality using APM tools, like New Relic and AppDynamics.
• Create a CI/CD environment: With Continuous Integration (CI), you have an automated software integration and unit test environment that enables continuous validation of development changes.  Continuous Deployment (CD) gives you an automated production test, release, and rollback environment that enables frequent, high-quality production releases. When combined with a culture of experimentation, continuous delivery equals competitive advantage.

Pythian’s DevOps team, through an unshakeable dedication to end-to-end automation, continuous testing, and the application of modern software development practices to cloud migrations, has been able to consistently deliver predictable results to our customers throughout the lifetime of migration projects.  Learn more about what we’ve done for customers on AWS by reading about our customer TinyCo.