Become a high-performing software organization

How do you become a high-performing software organization? It is more than just putting new tools in place to manage your CI/CD/CT pipeline. Doing so is like taking the skills you learned from riding a bike and applying them to Formula 1: it will not work. You need to apply new paradigms, take advantage of new capabilities, and shift your mindset to truly become more agile in your product-development process. This starts with having a clear understanding of where you are, and then simplifying your process by using new tools and creating new behaviors.

The goal is to have a strong, customer-centric product development cycle that seamlessly combines the efforts of product owners, stakeholders, engineers, quality specialists, as well as risk and compliance teams. When done well, this type of product-development process enables organizations to perform in a high velocity manner, with a focus on quality and value delivery.

Develop a process map

Although initially developed for manufacturing and Lean/Sigma, value-stream mapping (VSM) works well in understanding product flow in software development. The goal of VSM is to optimize the entire development process by understanding the role that each step plays, and then optimizing those steps so that each utilizes the least amount of resources (cost, time, and human), but still satisfies the customer’s value needs.

Step one: work backwards

Start by defining the problem or challenge you want to resolve. What is your goal: To go faster? To increase release velocity? Are customers concerned about quality or about the speed at which new features are released? Be clear about what you are designing your new process to achieve.

Step two: map the current state

Map out your entire product development process from a new idea through approval, design, development, testing, build, release, deployment, operation, and incident response. Your target should be a meaningful number of steps; enough to provide detail and clarity around your process. Typically this is twenty to twenty-five steps, but the goal is to combine detail with efficiency: do not spend the time documenting ninety-two steps and have too much detail, or have too little detail with just four or five steps.

Include:

1. All control points and compliance reviews and approvals (e.g., security and regulatory)
2. Project review boards (PRBs) that approve new ideas and allocate resources to projects
3. Portfolio reviews that coordinate larger pan-department projects
4. Regular status-reporting deliverables
5. Change control boards (CABs) that manage releases through production
6. Architectural review boards (ARBs) that approve new designs, architectures, and technologies
7. Any additional checkpoints or sign-offs throughout the process. Make sure to break apart what may appear to be a single, large step into discrete steps such as requirements, design, and development; and make sure to break apart the deliverable of each artifact into separate steps (e.g., visual design, data design, architectural design).

Identify the following for each step in the value stream:

1. The requestor or instigator who initiates the step (e.g., customer, product development, or software engineering)
2. The input provided into the step (e.g., work product, documentation, or ticket information)
3. The actor/role that is responsible for completing the step
4. The type of work performed (e.g., decision/sign off, activity, or control)
5. The output(s) for the step, including possible next steps depending on the pass/fail of the current step

Create a value-stream map with this information, then you can increase the level of detail as you seek out further optimization; for example, capturing the time necessary to complete a step, as well as the time between the steps. Real improvements can be made quickly once mapping begins. I find a simple swim-lane diagram that maps out the flow of steps across various actors to be straight forward, simple, and well received.

Our perception and ambition often do not reflect reality. You want to make sure you are not capturing the current state as you perceive it to be, or as you wish it to be, but rather as it actually is. This is often best done by direct observation of the existing processes, or through triangulation: capturing the state from multiple stakeholders and working through variations and misunderstandings to establish agreement.

Step three: identify and eliminate waste

Once your current process map is ready, you can start looking for waste. The goal is to make your product development more effective by reducing or completely eliminating steps that add waste to the process, or by completing necessary steps in a new and more efficient way.

Eliminate unnecessary steps

Is each step still necessary under your go forward approach? Is it possible a step was a holdover from the way things were previously done and is no longer needed? Perhaps it was necessary when you needed to purchase and install hardware, or when you worked with a particular GSI partner. Maybe it was needed when projects were approved in a more waterfall manner.

Eliminate handoffs

Handoffs cause a great deal of inefficiency. At Amazon we strive to eliminate handoffs as much as possible through our self-contained and autonomous two-pizza teams. When teams do not need to coordinate work then you eliminate the possibility of misunderstandings andf the result is fewer defects. Additionally, work can flow unimpeded when you don’t have to coordinate between teams. This means you no longer need to depend on well-intended, but ultimately flawed, portfolio-planning meetings to coordinate work.

If possible, eliminate handoffs by allowing the initiating team to perform as much work as possible. Sometimes this simply means allowing Team A to perform step four and step five, when previously they were not allowed to do step five. Sometimes this means enabling Team A to do step five through a self-service approach but Team B still maintains control. This might be necessary when there is a span-of-control issue. Finally, the capability could be automated so the step is automatically executed on the completion of the previous step.

What can be automated?

Almost every step of the product-development process can benefit from some form of automation, whether it is a fully integrated DevSecOps pipeline that automatically builds, tests, and deploys code; or the simple provisioning of new instances via infrastructure as code.

Shift left

The most expensive and disruptive time to find an issue with your software is near the end, but this is often where organizations invest in final oversight and perform security and compliance reviews. High performing organizations conduct these control steps as early in the process as possible and integrate them with the development process. For example, pre-configured instance types can have security settings established as part of a common image.

For other controls where validation must occur, the goal is to find the earliest point during the process where a control could be applied. For instance, how can you reimagine your requirements process so it meets the needs of your engineering teams, and the needs of your compliance teams at the same time? How can you rethink your approach to testing and test matrixes so that the testing goals your engineers meet also provide the evidentiary proof that your compliance team requires?

Eliminate oversight boards

I have built and led my fair share of PRBs, CABs, and ARBs. All had good intentions, but I am not sure if any of them really achieved what they were designed to achieve. In today’s agile, product-development process they usually slow things down and add more work.

A good approach I have used is to work with teams to ask (at the end of a CAB, PRB, or ARB meeting)  “Why did we need this meeting, and what could we have had to have done differently in the future to not need this meeting?” Out of this comes a list of items that need to be addressed so that you can eliminate the need for the meeting. This may require further refinements to the product-development process, helping people become more confident in how releases are managed, or in helping people appreciate that there is neither the need nor the opportunity to keep everyone informed of all of the planned releases.

Often these meetings are holdovers from the way things have been done in the past. They exist just to make people feel comfortable. Ask yourself how many times a change has been rejected at a recent CAB? My guess is infrequently. CABs existed to coordinate large releases back when software was architected as a large monolith—where the “blast radius” for a release was significant, and therefore required a lot of testing and planning. If your organization uses micro-services and APIs for everything, and your application programming interfaces (APIs) have guaranteed contracts, there is no need for the CAB to notify other teams of a pending release, as the release should have no impact on them.

Map out your new process

Once you have evaluated each step in your value stream, map out your new product-development process and begin working on an implementation plan. Likely, it will take a number of steps to implement some of the new self-service steps and automation necessary to deliver your new process. Take time to train people on the process. Tabletop exercises are a good approach and not only serve to teach, but also provide a great opportunity to learn how the process could further be improved from the boots on the ground.

You should continue to revisit your product development process, because any process is prone to decay. Do it regularly to ensure that:

1. People are following the process
2. The process works as designed
3. You are able to take advantage of any new developments in approach, technology, or controls.