Make Automation a Key Part of Your Cloud Strategy

“As machines become more and more efficient and perfect, so it will become clear that imperfection is the greatness of man.” — Ernst Fischer

Last week I returned to the east coast after a productive three weeks on the road. I was able to attend and speak at the AWS Summits in San Francisco, London, and Sydney and meet with several dozen executives at different stages of their cloud journey. It was encouraging to see how IT is evolving across every industry around the world while also learning about some of the regulatory nuances between the US, Europe, and Australia. I’m fascinated by the regional differences, and am looking forward to learning more about them during my trip to Asia next month. I’ll look to author a post that details some of these nuances when I get back in June. In the mean time, there were at least two themes that consistently came up in my conversations abroad that I’ve detailed over two posts. This one will focus on the importance of automation, and the other on avoiding analysis paralysis.

Automation takes the value an organization gets from embracing the cloud to the next level. The cloud removes much of the undifferentiated heavy lifting associated with managing traditional IT. Automation offers an organization additional benefits that extend into many different IT disciplines. Because these benefits spread throughout so many functions inside of IT, it’s not surprising that I find myself weaving automation into conversations that start from a number of different topics.

There are a number of different ways to think about automation. For the purposes of this post, I’m referring to being able to reliably reproduce related IT resources by executing code of some kind. A practical example underpins what’s become known as continuous integration/deployment. When an application can be reliably reproduced — built, deployed, and perhaps rolled back if something goes wrong — teams can be much more confident when they deploy changes. In most cases this means they’ll do it faster and more often. There are a number of mature tools that exist in the market that are helping companies do this — Chef, Puppet, Ansible, AWS CloudFormation, and others. While you should always find the best tool for the job, embracing and implementing automated systems is far more important that which tool you use to do it.

I find myself referring to a handful of benefits that automation provides when answering questions such as “How can I save money with the Cloud?”, “How do I make my applications secure?”, “How do I avoid being locked into one platform?”, “How can I move faster?”, and many others. These benefits are, in no particular order:

Efficiency. As a CIO, one of my primary responsibilities was to be able to focus an increasing percentage of my resources on initiatives that drove revenue into the business. This was our main motivation for using cloud services — by reducing the amount of time we spent managing infrastructure we could repurpose those resources toward product development. By automating repeated tasks such as operating system patches, application deployments, firewall changes, and monitoring/alerting configurations we were able to devote more cycles to the development of features.

Elasticity. This is one of the most beneficial features of a cloud platform. Automated systems that can be reproduced using scripts can leverage features like AWS auto scaling and fluctuate capacity with demand. In my experience capacity planning is generally very hard to do, and getting it wrong can be a large waste of capital and/or very frustrating to your customers. Elasticity is one of the quintessential benefits of cloud computing, and will never cease to amaze me after having poorly planned capacity for nearly every system I’ve built without it.

Portability. When a system can be reliably reproduced on one architecture or infrastructure, it will be much easier to reproduce it on another. If for whatever reason you need to change where a system is hosted, it will be far less effort and to modify the automations scripts than if you had to customize each platform manually.

Security. Automating good security practices into your systems makes it much more likely that your system will be uniform as it scales, removes the element of human error creating an exposure, and allows you to roll out security improvements (like any feature) quickly and with confidence. Some of the better-architected systems I’ve seen leverage automation in phases. During each phase a specific action is performed — updating an operating system, downloading code from an internal repository, bringing the application up on the Internet, while only allowing the connectivity needed for each phase. This helps make sure traffic coming in from the Internet doesn’t have connectivity into corporate systems.

Auditability. Systems built through automation scripts tend to behave more predictably and have more informative logging. The sequence of actions taken for anything that’s automated is by its very nature documented by the code that performs those actions. This makes it much easier to describe to others — your team, business stakeholders, auditors, and regulators — who performed what action and when in your systems.

Recoverability. There are at least two factors that work in your favor when building automated systems when it comes to recovery. First of all, as you gain experience with automating common actions, it will become clear how to implement automated health checks and common QA actions. Automation can then be used to roll back systems in flight that don’t pass these checks. Secondly, as the friction to make changes to an environment is reduced to zero, you can become more aggressive with how you push out changes. It becomes less costly and time consuming to roll something out and either roll it back or roll out a patch on top of it than it does spending days running manual QA or debating the risks.

Do you have any other reasons? I’d love to hear about them. Drop me a line.

Keep building,
–Stephen
@stephenorban
orbans@amazon.com