Top-Down Transformation: Shu-Ha-Ri to Go Beyond Command-and-Control

In a previous blog post, I talked about how to drive top-down change in an enterprise, and how to take advantage of command-and-control authority even in an Agile setting. In this post, I’ll talk about a particular technique that draws on authority. It is the change management approach called “Shu-ha-ri,” a concept drawn from the martial arts, where authority is derived from the respect a learner has for someone who has already mastered the discipline. Alistair Cockburn described the use of this technique in an Agile setting in his book Agile Software Development in 2001 and in a later blog post. [1]

Shu-ha-ri are the Japanese characters for “hold, break, and leave,” or “learn, detach, and transcend.” The framework describes three phases that learners pass through. In the Shu phase, the learner does what the master says, exactly and without questioning. The learner focuses on the task rather than the theory. In the Ha phase, the learner starts to understand the principles and the theory underlying those practices and draws upon additional sources of information to inform the practice. Finally, in the Ri phase, the learner has internalized the principles, breaks free of the specific practices, begins learning from himself or herself, and creates new practices that are aligned with the principles.

With this model in mind, the transformational leader can begin by setting up Agile objectives as policy and then (temporarily) requiring a set of practices that will accomplish those goals.

Employees are required to follow the practices without necessarily understanding them. The understanding comes naturally over time, as the good practices lead to better results. The leader can coach the employees through these phases, guiding the employee along the learning journey. Finally, the employee will have thoroughly internalized the principles and can now innovate around them and teach others.

For various reasons, I decided that this was the best way for me to conduct the transformation at USCIS (US Citizenship and Immigration Services). Among other things, it was a convenient approach for aligning the various oversight bodies with the transformation. I began by writing a piece of government policy (Management Instruction CIS-OIT-001) that required all projects within the IT organization to follow a set of practices, which were essentially the practices of Scrum. These practices, as I framed them, were: frequent delivery, time-boxed iterations, user stories, product owners, release planning, iteration reviews, retrospectives, and continuous testing. Later, as we learned more, I replaced this with a new Management Instruction that focused more on principles and goals and was more oriented toward DevOps and Lean flow-based practices.

The policies also set up a watchdog, or auditing arm (Independent Verification and Validation, or IV&V), to make sure the teams were using these practices. Over time, the teams became comfortable with these techniques and began to consider them normal. Eventually I realized that they had become creative with Agile ideas in ways that I hadn’t expected.

As an example, let me tell you about our QA organization. Before we began the transformation, QA’s job was to review each system before it was deployed. If QA found quality issues, they would forbid deployment and have the developers go back and fix the defects. QA acted as a gatekeeper to production. They were proud of this mission—in their minds this was the control that ensured that only high-quality product was deployed to users.

When we began the transformation, I told QA that their role was changing. Their mission would remain the same: they were to ensure that only high-quality product was deployed to users. But they were no longer going to be given the opportunity to review the system just before it went to production or to hold it up from launch. Instead, they would have to find a way to ensure the quality of the product as it was being built. “If you need to hold up a system from being launched,” I told them, “that means that you’ve misunderstood your job. Its quality should have already been good enough for production.”

The technical term to describe QA’s reaction, I think, is that they “freaked out.” They warned me that this was a disastrous idea, that quality would suffer, and so on. But then their leader went away and came back with a number of suggestions for practices, which we discussed and institutionalized to begin the Shu phase of learning. QA would attend team planning sessions and reviews, and ask hard questions when necessary. They would periodically audit a sample of the automated tests that developers were creating to make sure they were testing the right things. They would produce health metrics on the development process and track the number of escaped defects and their root causes. And they would oversee that IV&V process that made sure the Agile “rules” were being followed. All of these practices would be used to help the teams improve their practices, not to “tattle” on teams to me or other managers.

The new QA practices became a requirement, and the QA folks became really good at them. Quality improved, as did deployment frequency. Systemic problems were identified and fixed. One day I was at a meeting with some IT folks from another agency sharing good practices. A number of our QA folks were there as well. Every time the people from the other agency said something that was not Agile, I could see the QA people wince and look at them like they were crazy. That’s how I knew that we had entered the Ha phase. The QA team had internalized and understood the principles, and could immediately recognize anything that diverged from them. Certain ideas just no longer seemed right.

One day QA asked for a meeting with me to propose a new idea. “If we are in charge of quality,” they said, “we think we should also be in charge of training the developers. Can you move training into our organization?” Now, I had never heard of training being under QA, and would never have thought of it myself. But they were absolutely right. I had charged them with ensuring that systems were built with high quality, and the best way to do that was to make sure that the developers thoroughly understood how to build with quality. They had entered the Ri phase. They had thrown away the traditional ideas about QA and even the new ideas I had given them about QA—they were actually creating good practices themselves.

Is Shu-ha-ri really a command-and-control approach? In martial arts practice, perhaps not—the authority really comes from the respect that the learner has for the master rather than from some organizational power that is compelling behavior. In the example above, although I would like to believe that the employees might have had some respect for me as the ninja IT-master that I was, in truth they followed the practices because I was senior in the hierarchy. But the effect was arguably similar—once the employees started exercising the practices they started to internalize their logic.

After the Shu stage, though, the leader’s role is less command-and-control and more one of a coach, helping the employees understand the ins and outs of the practices, expanding their repertoire of practices, giving them opportunities to learn and test out ideas, and connecting them with others they can learn from.

It’s intriguing to see that command-and-control authority can play a role in fostering a servant leadership environment. As I explained in my previous post, it seems that the authority of a senior leader may be an asset that can be used—judo-like—as a way to get enterprise transformation moving. The Shu-Ha-Ri model gives us a framework for making sure that what starts as command-and-control ends as team empowerment.

–Mark
@schwartz_cio
A Seat at the Table: IT Leadership in the Age of Agility
The Art of Business Value

[1] As a side note, I am always careful about using Japanese concepts in areas they weren’t meant to apply—please don’t take this as an explanation of how the term is actually used in Aikido!