Organizing for Insights

Advancements in artificial intelligence (AI), machine learning (ML), and big data technologies enable organizations to harness the troves of data being captured (or which were previously thrown away), bringing new insights, automating processes, and even creating new products, services, and businesses. AI, ML, and the associated data (powered by the AWS cloud) is a critical component of any digital transformation. I’ve written a few posts as to why organizations need to adopt these technologies and what companies should expect. However, with any major technological shift the challenge often has more to do with the organization, its processes and governance, and the people dimension than the technology itself. In this post, I’ll introduce common patterns I’ve seen around organization, roles, and processes related to establishing an advanced analytics capability.

I see direct parallels to the cloud journey when establishing an advanced analytics capability. To go from strategy and intent to meaningful progress, one of the first steps is to identify and choose a leader and a team to spearhead the change, and by setting up an analytics center of excellence (COE). The team typically starts small, with a few cross-functional roles to bootstrap the team, which will then grow to service more and more needs. Many large enterprises already have established business intelligence or reporting shared services organizations. Those organizations can seed the analytics COE with the technical and business roles. Similar to IT infrastructure organizations, they should not only supply the talent, they should be a key driver and sponsor for the effort because over time these reporting shared services organizations will need to evolve to either adapt or become part of the analytics COE. The starter roles are often data engineers and architects, business intelligence analysts, and data scientists. The group needs to be led by someone who can work across multiple organizations, business units, and back office groups (e.g., finance) as well as IT.

At Amazon, anything we build starts with working backwards from the customer’s needs. Customers who have ever-changing needs, increasing standards, or who can never be satisfied turn out to be a wonderful source of innovation. One of the primary mental shifts that organizations need to make is moving from an attitude of “you must use our reporting solution and you will like it” to “what are your analytics needs and how can we help enable you?” Shared service reporting organizations are often report pushers and not positioned to answer the difficult questions that come from employees, business leads, and customers. Therefore, when establishing a new analytics COE it’s important to establish tenets for the group which will set expectations for how the group will act and make decisions.

The analytics COE will need to service two types of customers. The first group of customers are the data and analytics consumers: the decision makers, data scientists, business intelligence (BI) analysts, and developers. These customers typically care about the ability to quickly access insights and data, and the quality of the tools and services available to them to process and present data. The second group of customers are the data producers: the owners of applications, infrastructure, and devices who will supply the data into the platform. These customers need services like the ability to easily publish their data into the analytics platform and define a data contract, which includes the domain model of the data, frequency of refresh, and definition of policies (e.g., security policy for who can access their data).

It is important to understand that the COE is serving these two customer types, because the analytics capability and the platform needs to service both types of customers, and if their needs are not met then the analytics effort will not deliver the business value. Therefore, it is critical to have a mechanism to capture the needs of these two types of customers across a potentially very large and diverse set of business units and personas. Some organizations set up advisory boards or work with a few key stakeholders to drive the need. There is no single correct answer, but having a mechanism to capture the voice of the customer(s) and prioritizing their needs is critical.

I shared a post that proposed a new mental model for how to think about the AWS cloud as a superstore for digital services. Any great retail experience gives customers a combination of service, selection, value, and convenience. An analytics COE services and presents a specialized set of cloud services focused on meeting analytics needs. In the past, reporting and BI organizations often provided one solution to fit everyone’s needs (a one size fits all strategy). In this era of rapidly evolving technologies in big data, rich visualizations, automated decision making, artificial intelligence, and machine learning it’s just not possible to have a single technology stack. It’s not just about having the latest tools in the belt; it’s about making it easy for your customers (producers or consumers) to get what they need.

COE’s run the risk of becoming concierge services, which can be good for certain types of requests, but the COE can quickly get overwhelmed and back logged with requests if there’s not scalable self-service mechanisms and transparent prioritization and governance processes in place. Analytics COE’s need to engineer and architect a self-service, secure, operable, and scalable data platform with an ever-evolving ecosystem of technologies to process, analyze, and present insights.

I hope this post gives you the motivation to organize around developing insights for your business with a customer-oriented focus to take advantage of all the services that AWS provides and become a data driven organization.

Never stop innovating,

Joe

@chunjx
http://aws.amazon.com/enterprise/