Increasing Analyst Productivity and Data Trust with Alation Cloud Service

By Dilip Rajan, Sr. Partner Solutions Architect – AWS
By Jason Lim, Cloud Marketing Director – Alation
By Kish Galapati, Solutions Architect – Alation

Alation

For a business to effectively prepare for unforeseen circumstances and unpredictability, it requires a healthy and robust data culture inside the organization.

At the same time, ever-evolving laws, regulations, and corporate policies have added pressure to comply and manage data properly. Without a data culture, organizations struggle to balance the benefits of using data to drive business growth versus protecting the business against the risks of data misuse or data insecurity.

According to The Alation State of Data Culture Report (Q3 2021), 63% of data professionals say their data strategy is more likely to be focused on enabling business growth versus protecting the business. That puts governance, privacy, security, and other critical data issues secondary to growth.

Alation is an AWS Partner that provides artificial intelligence-driven data search and discovery, governance, and analytics capabilities to help organizations foster a data culture.

In this post, we will explain how Alation makes data users more effective and productive, and how Alation can be used to govern data to ensure its proper usage.

Alation Data Catalog and Cloud Service

Alation Data Catalog provides a single place to find, understand, and collaborate on data by removing barriers between users and the huge volumes of data, complex environments, and data and organizational silos.

It does this by creating a single metadata repository for information across an organization’s databases, data lakes, file systems, documentation, SQL queries, business intelligence (BI) tools, and more. This eases data search and discovery, data literacy, data governance, self-service analytics, cloud transformation, and digital transformation while helping to ensure privacy, reduce risk, and improve compliance.

Alation Cloud Service gives organizations the power of Alation Data Catalog through a solution hosted and managed by Alation on Amazon Web Services (AWS). This deployment option is more straightforward compared to self-hosting or managing Alation on premises.

Vista, a Cimpress Company and Alation Cloud Service customer, is the design and marketing partner to millions of small businesses all over the world. From posting on social, to refreshing your logo, even hosting an event or a grand opening, Vista can help. A key objective of using Alation is to create a self-service data marketplace to connect data producers to data consumers.

“Alation Cloud Service provides the full benefits of the Alation Data Catalog while leveraging new catalog features faster with zero administrative overhead,” says Oliver Bauer, Sr. Director at Vista. “This allows us to focus on our core business innovation and better serve our customers.”

The diagram below highlights the architecture for how Alation interacts seamlessly across AWS and on-premises data sources as a unified catalog.

Figure 1 – Architecture diagram showing how Alation Cloud Service runs on AWS.

Analyst Productivity is Critical

According to The Alation State of Data Culture Report, three in five data leaders cite a lack of data democratization and/or organizational silos as challenges when using data to drive business value. When data is sprawled across businesses, departments, and regions, it becomes difficult to find, understand, and use.

The consequences of this lack of visibility can be profound. Here are a few examples with significant productivity implications:

• If an analysis has been done but not published for others to see, data analysts and data scientists will end up redoing the work.
• If data definitions are not well documented, incorrect assumptions will be made.
• If there’s a question about the data but no way to identify who to ask, people will get frustrated by spending days or weeks searching for them and delaying results.
• If data is inaccurate, incomplete, or out of date but not flagged as such, people will inadvertently use bad data.

These scenarios result in lost analyst productivity, poor decision making, indecision, and analysts who are unable to generate optimal business value from data.

How Alation Boosts Analyst Productivity

Alation solves analyst productivity problems by creating a community of data citizens who collaborate around a single repository called a data catalog. This catalog constantly indexes the data environment so the current state is always reflected.

Alation uses a combination of machine learning (ML) and human curation to add helpful context to the data, such as translating technical names to business-friendly names, showing what data is most popular, and highlighting the most active users. This rich context helps analysts interpret data and know who to go to with questions.

Analysts can publish their SQL queries and projects back to the catalog for others to see and reuse, and collaborate alongside the data by asking questions or giving answers.

With Alation, many customers have experienced significant improvements in analyst productivity. Specific examples include:

• Finding the right data in two hours instead of 80 hours.
• Saving 325 workdays over a year by reusing existing SQL queries.
• Saving 92 workdays in analyst onboarding over a year, by avoiding installation and configuration of multiple SQL query tools.

More productive analysts translate into valuable time and money saved for the business, in addition to providing benefits like higher employee satisfaction and morale. The last two examples of productivity metrics above were calculated by rigorous data science research.

Accelerate the Time to Trustworthy, Protected Data

Data governance must be a requirement, not an option. Every organization should strive to deliver data that business leaders and data modelers can trust.

Unfortunately, traditional command and control style governance initiatives often fail. They fail because they focus on process and outcomes of compliance—rather than efficiency, scalability, or ease of use of data for the average human.

As a result, organizations are often left with a decaying business glossary. How people actually consume data is often a forgotten part of the data governance equation.

Alation fundamentally changes the equation by making data governance a core capability for every organization’s data operation. The solution is built on three principles:

• People-centric and non-invasive
• Continual improvement
• Autonomous operation

Similarly, Alation’s differentiation has three components:

• Alation uniquely uses AI/ML and concepts of continuous improvement to shorten the time to governed, discovered, and protected data.
• Maps the product to best practice, non-invasive data governance.
• Provides a blueprint for customers to create a data governance solution.

Advanced Data Governance Capabilities

In September 2021, Alation launched an intelligent Data Governance App that provides broadened platform governance functionality. This includes a policy center, workflow, enhanced collaboration, stewardship and governance dashboards, and policy usage monitoring.

The app delivers autonomous data governance through the Behavioral Analysis Engine (BAE). Unlike difficult, tedious, and manual data governance, Alation automates many of the stewardship processes. It automatically determines the best candidates to become data stewards, collects and enriches data, ensures governance of the right data first, and monitors and measures rule violations.

Customers have experienced great success with Alation for data governance. Before it used Alation, a leading video streaming service required a month to determine what data is SOX compliant. With Alation, it now takes seven seconds.

Using Alation, a leading bank can now reduce risk and stay in compliance with BCBS 239 requirements by maintaining and classifying audit standards. A life insurance company is using Alation to automate data lineage, perform root cause analysis, and increase audit compliance.

How Alation Connects and Indexes AWS Sources

Alation can connect to a wide range of data sources through native connectors, a software developer kit (SDK), or APIs. Alation also supports many popular AWS sources, including Amazon Redshift, AWS Glue, and Amazon Simple Storage Service (Amazon S3).

Using Amazon Redshift as an example, let’s look at how Alation can enhance analyst productivity while ensuring strong data governance.

After natively connecting to Amazon Redshift, Alation scans and indexes all of the schemas, tables, and columns to populate the data catalog. To augment the contextual richness, Alation uses a process called Query Log Processing to provide valuable insights, such as how much data is used (popularity) and who are the likely subject matter experts (top users).

Moreover, Alation uses clever machine learning to translate technical titles into more human-readable titles. These capabilities help analysts be more productive by answering questions like:

• “What does this table or column really mean?”
• “Which data should I look at first?”
• “Who should I ask if I have a question about this data?”

This helps to reduce onboarding time by empowering people to self-serve data, and it automates the stewardship process of describing data for others to comprehend.

Figure 2 – Alation Table Catalog page.

Query Log Processing automatically produces interactive and detailed lineage graphs. These help users understand the relationship between data assets; for example, from the Amazon Redshift data warehouse to a business intelligence report.

Additional useful information, such as dataflow object extract, transform, load (ETL) transformations can also be added to the lineage graph via API.

Lineage is valuable in providing data governance by helping to answer questions such as:

• “Where did this data come from?”
• “How did this data get created?”
• “Which data is impacted if another piece of data changes or has a problem?”

In answering these questions, organizations can trust their data when managing complex compliance and regulatory requirements.

Figure 3 – Alation Lineage graph.

MercardoLibre operates major online marketplaces and auction sites for the Latin American region. With Alation, MercardoLibre is able to democratize certified data, including Amazon S3, by setting up self-service analytics enterprise wide.

“Alation provides our more than 7,000 users with independent access to information, data, and queries across our hybrid data environment,” says Adrian Quilis, Sr. Director of Business Intelligence at MercadoLibre. “Alation and AWS are critical to helping us achieve our business initiatives.”

Summary

In this post, we have looked at the need for organizations to cultivate a data culture and how Alation supports that journey.

We covered what Alation Data Catalog is and explained that Alation Cloud Service is hosted on AWS and managed by Alation as a service. We highlighted two essential pillars of data culture—data search and discovery, and data governance—and illustrated how Alation works with Amazon Redshift.

If you are interested in learning more about Alation, please contact your AWS representative or reach out to Alation directly at alation.com.

.

.

Alation – AWS Partner Spotlight

Alation is an AWS Partner that provides artificial intelligence-driven data search and discovery, governance, and analytics capabilities to help organizations foster a data culture.

Contact Alation | Partner Overview

*Already worked with Alation? Rate the Partner

*To review an AWS Partner, you must be a customer that has worked with them directly on a project.