How Can You Build a Culture of Experimentation?

In my conversations with hundreds of enterprise executives worldwide, I have found that they like the idea of moving fast and innovating but find it difficult to accomplish within their corporate legacy and constraints. Innovating quickly is especially urgent in light of generative AI’s potential. But to increase innovation, you need to build a culture where experimentation is not just tolerated but the norm.

This post explains effective ways to gradually build a culture that encourages experimentation with low cost, low risk, low friction, and large payoffs for the enterprise. I saw these techniques work at NASA’s Jet Propulsion Laboratory (JPL) when I was their IT Chief Technology and Innovation Officer. JPL’s culture of experimentation has led their teams to use the cloud for today’s space missions, helping them make faster decisions and pivot when needed. Rapid experimentation also led to the infusion of many new technologies, such as augmented reality, 3D printing, data visualization, and democratized machine learning at scale. Existing personnel, interns, and early career hires introduced these innovations quickly and inexpensively, which helped the entire organization and paid significant dividends for the experimenters’ careers.

To build a culture of experimentation, first define what an experiment is and is not in your organization. This helps set expectations and lower the fear that your proposed experiment could slip into live production without adequate development and controls.

Second, pick a business problem where a potential solution can be demonstrated within two weeks. Work backward from a customer’s needs and fall in love with a real business problem, not a technology or particular solution. Solutions evolve over time. To determine success metrics that lead to insights, have the experimenters give you a one- or two-page write-up that describes their desired outcome and update it with the results after each experiment. Save these business problems in a library of use cases that future experimenters can access for training and iteration. This is a terrific way for employees to learn new skills while solving a small but real business problem.

Third, find the experimenters across your entire business and functional organizations. Many organizations are surprised to find they have interested and knowledgeable people; they just need approval and encouragement to try new things. For example, those well-versed in prescriptive analytics easily adopt AI, machine learning, and generative AI.

Fourth, use a cross-functional approach. Assign a two-to-four-person team consisting of at least one IT employee, one end user from an interested business unit, and a cybersecurity professional to offer advice and awareness.

Fifth, measure the success of your experiments. Don’t fixate on measuring return on investment (ROI); you’ll spend more time and money (and lose momentum) creating an ROI document than it will take to implement the experiment. Measure return on attention (ROA) from the end user community; i.e., the level of interest from affected end users. Score ROA from 1–10, with 10 being 100 percent active, positive attention. At JPL we knew that if we earned an eight from at least two end user communities, we would get the funds for investment. Then we could measure ROI and iterate toward perfection (or close enough). I have worked with other companies to successfully develop a similar approach. It helps unlock creativity from the organizational edge, where the people who are closest to the problem—and often the solution—live.

Sixth, tell the story of the experiment. Once you have solved business problems with emerging technology, hold in-person demos and consider holding an open house where you show multiple demos. Create a self-service viewing capability by recording annotated demos. It’s important to set expectations that you are not a professional videographer to avoid criticism that your initiative is a gold-plated science experiment. Remember to give credit to the end user (business) participants, especially since other end user communities are most likely to accept their opinions, and you will create a demand for participation in future experiments. Use social media and awards programs to benefit your practitioners.

If you follow this approach, you won’t need a large budget to get started. Too much money over-complicates the solution, and too much time loses key implementers due to emerging business-critical issues. Start with only two to four people for two weeks to establish high cohesion and focus. The most successful enterprises establish a small budget that the CIO/CTO or line of business can spend without asking for permission or forgiveness. For those who are bold, promise a 10x ROI. It helps to get the budget; at JPL we had no problem meeting that goal!

You should also avoid setting up a “one-and-only” innovation group as it can foster elitism and resentment. Innovate together with the end users and give away credit freely. Avoid outsourcing thought leadership; partner with specific technical expertise and maintain the vision yourself. You can make experimentation easier and faster with automation that includes compliance controls. Consider experimenting below the radar and going public only once it’s successful; that reduces fear. Remind stakeholders that you are conducting a low-risk, low-cost experiment with potentially unlimited upside that can transform your business.

Success in a culture of experimentation is all about the business outcome. Each experiment should deliver a story and be easy to undo, which reduces risk and objections. The only real failure in an experiment is that your organization doesn’t learn anything, and we’ve never seen that happen. The steps above are fairly easy to try in an existing enterprise as you don’t have to change everything across the entire organization at once. If you can improve a business process by 10 percent again and again, you will effect real change in your company. Your organization will be a fun place to work, which will help with internal ideation and recruiting, retraining, and retaining talent. And that’s a worthy goal indeed.

Please let us know your thoughts, results, and ideas.

Live long and experiment!