Enterprise DevOps: What to Expect When You’re DevOps’ing

© Andrew Hurley https://www.flickr.com/photos/15717926@N04/6254409229

“You cannot create experience, you must undergo it” -Albert Camus

I’m wrapping up my DevOps in the Enterprise series with a piece on what to expect once your organization has some DevOps experience. These are just a few of the things I’ve experienced or seen in organizations that embrace automation, customer-service-oriented IT, and a run what you build mentality.

Expect to grow your DevOps practice slowly

Like most things worth doing, implementing a culture of DevOps takes time. I encourage anyone going down this path to start small and deliberately. Measure the impact of each organizational change you make, embrace what works, and celebrate the things you learn from what doesn’t work. This helps the organization grow toward a culture of continuous improvement. The most challenging part of the whole process is getting started. As you become more and more comfortable with a DevOps culture, the unique challenges your organization may face should become obvious, and the solutions for those challenges won’t be far behind.

When we implemented DevOps at Dow Jones, we started with just four people and grew the team slowly by adding one or two more members to the team from other areas of IT each month. This allowed us to build up some experience and best practices, and grow in line with the number of projects to which we were applying DevOps. I wouldn’t recommend trying to grow much faster than this. Slow and deliberate growth allows you to set realistic expectations with your stakeholders, which includes your team, for how quickly things will change. It also allows you to consume resources in proportion to the overall business benefit, which may help you navigate around unnecessary politics surrounding resource allocation.

Roughly 18 months into our Journey, after we felt we had built a large enough inventory of best practices, automation, and governance models, we became comfortable moving the majority of our infrastructure resources into our DevOps team. Our goal with this change was to have these individuals adapt their traditional roles to the DevOps mentality using the experiences the DevOps team built up over time. People didn’t drastically alter how they did their jobs overnight. They did, however, start to change, little by little, the way they managed all of our systems.

Be open minded about where and how you apply DevOps

There is no one-size-fits-all way to implement and gain experience with DevOps. Every organization is unique. Not everything needs to — or should — change. Be prepared to measure what works and what doesn’t. Find a way to tie your DevOps mentality and/or team to a business benefit — which can come from any part of IT. The industry often emphasizes how DevOps and a culture of innovation applies to product development, but I have seen these practices be just as applicable to back-office, end-user computing, and other parts of IT. Being open minded about the types of projects you take on will allow you to stay focused on nurturing your DevOps team and understanding how it can work best for you.

Here are some of the benefits I’ve seen organizations celebrate as they mature a culture of DevOps:

Constant releases. A DevOps culture should be conducive to smaller changes being made more frequently. I talk about this in my run what you build post, where I list several benefits that can be summarized as greater efficiency, more resources aligned to business needs, and better operational excellence. All of these things lead to a better experience for your customers (internal or external). Be prepared to manage the expectations of your business stakeholders as this happens, and don’t get too far ahead of them. Be sensitive to the fact that your stakeholders may view a constantly changing product or environment as a risk. It will take time and maturity to build and prove the mechanisms that you need to do this responsibly. Building trust is essential, and building trust to do something new takes time. There’s no glory in being the only one at the finish line.

Globally distributed apps. Watching an application scale up and down across different regions of the world as each time zone conducts its business is one of the most rewarding experiences I’ve had as an IT executive. As your DevOps team understands how to automate and manage a fleet of resources across different regions, it will become straightforward to distribute your applications globally. Pushing your services closer to customers will lower latency, making your system more efficient, cost effective, and, of course, delight your customer. As your mastery of DevOps improves, it will become increasingly easy to distribute your applications globally.

The data center migration. Everything IT does should be driven by the business. And when an IT exec can take something considered a traditional IT initiative and turn it into a real business case, they start to cement their place as a business executive overseeing IT, rather than simply an IT exec. Using your inventory of automation and globally distributed applications, you can develop compelling business cases to migrate some or all of your data centers to the cloud. I’ve seen this happen with increasing frequency over the course of the past year.

I experienced this after about 10 months of running a DevOps team at Dow Jones. We were leasing a data center in Hong Kong that was going to be closed in just a few months time. We had to quickly find somewhere else to host our infrastructure. I felt that we had a strong enough momentum with our DevOps practice and cloud expertise that it would have been a shame to take a step backwards and lock ourselves into another data center commitment.

After some initial resistance, the team found a way to engineer around all the perceived roadblocks, and we completed a full data center migration to AWS in six weeks. While this particular deployment looks much differently today than it did when we completed the migration, I think it remains a great testament to what happens as you build expertise and manage expectations over time. There’s no way we would have been able to achieve that migration without the experience we gained leading up to it.

Have stories of your own to share? I’d love to hear them.

Keep building,
–Stephen
@stephenorban
orbans@amazon.com