The strategic power of data, enabled by AWS Partner data-led migrations

Data has become a strategic asset. Data gives insight. Data is now the starting point for how a company, or agency, or government, conducts business. With the right tools, data becomes predictive and makes us agile.

A number of trends have made cloud-hosted data even more important. Internet of Things (IoT)-connected devices, apps, and systems now generate more data than ever before. According to IOT Business News and IDC, 56 billion connected devices will generate 79 zettabytes of data by 2025. Nearly 30 percent of data will require real-time processing. And with the cloud driving down the cost of storage, customers no longer need to decide what data to keep and what to throw away. Amazon Web Services (AWS) dropped the storage price per gigabit by 83 percent from 2008 to 2019.

With the cloud’s ability to make formerly separated and siloed data available to analysis tools, organizations can now more easily analyze their data to gain insights in a variety of different ways.

Capturing relevant business insights fuels innovation, which leads to competitive advantages.

AWS Partners can help

Using data-led migrations and advanced data analytics solutions, AWS Partners create repeatable and scalable solutions that provide increased operational efficiencies and profitability for companies, agencies, and governments. The three steps to gaining the full value of data are: migrate data to a data lake in the cloud, set up an analytics engine, and leverage artificial intelligence (AI) technologies, including machine learning (ML) and deep learning.

For data to have gravity in an organization, moving data to the cloud enables the magnitude of data to be manageable, secure, and actionable.

Migrate to a data lake: To store large datasets, you need highly available, secure, and flexible environments. A data lake holds unprocessed data in its rawest form. It stores structured and unstructured data at any scale, enabling analysis across formerly siloed data.

There are lots of steps involved in getting data out of silos and making it usable by analytics tools—and AWS Partners can help. Storage needs to be set up so that data can be moved. Some of the data needs to be curated—cleaned, enriched, and transformed—so it can act as the single source of truth. Security policies need to be configured to protect information.

AWS Partner NorthBay helped Evisions, which provides administration software to higher education, create an AWS-based transformative platform that features a new data ingestion layer, a software as a service data lake, and a data consumption and access application. These solutions allow Evisions’ customers to expand data access more broadly across their campuses and automate repeatable tasks.

Some AWS customers have migrated four to six times faster (2-3 months versus 12 months) and scaled more effectively when focused on data-led versus application-led migrations. For example, within 4 weeks Palantir migrated data used by the US Department of Veterans Affairs to AWS to track and analyze COVID-19 outbreak areas and make timely decisions using supply chain, hospital inventory, and social services data.

Set up analytics: Once the data is stored, you needs to analyze it to get value from it. Analytics tools help provide insight from data. They search, explore, filter, aggregate, and visualize data in near real-time for application monitoring, log analytics, and clickstream analytics.

The Financial Industry Regulatory Authority’s (FINRA) approach to catch fraud more effectively, through their work with AWS Partner Splunk, is a clear example of how data-led migrations enable impactful data insight. Every day in the US, as many as 100 billion securities market financial transactions take place, involving billions of investors’ dollars. FINRA oversees market integrity for these transactions, which they collect from 170 applications, capturing, indexing and correlating exabytes of data real-time in the AWS Cloud.

Arkhotech, an Advanced Partner based Chile, Peru, and Columbia, enables businesses to take advantage of their own data, offering services in application modernization, well-architected solutions, and data analytics. When distributed and disjointed data from different government institutions slowed Subsecretaria de Prevencion del Delito of Chile (SPD) from meeting their mission of crime prevention, Arkhotech worked with SPD to migrate nine sources of data. After migrating the data to a data lake on AWS, Arkhotech used data analytics to make data reporting and public data publishing simpler and supplied data visualization tools. Information availability time decreased from two weeks to one day. The analytics also enable SPD to gain accurate and timely insights that could drive business decisions, such as resource allocation, citizen protection needs, and reveal operational inefficiencies.

Leverage AI and ML: The business landscape and the corresponding rules regarding data are changing so frequently that systems need to be intelligent and agile enough to adapt to these changes at a quicker pace. This is why ML has become a necessary component of advanced data analytics systems. Machine learning systems evaluate data, assess the quality, predict missing inputs, and provide recommendations.

The City of Los Angeles, the second most populated city in the US, generates 240 million security records daily from at least 40 different agencies. AWS Partner Splunk’s implementation of real-time analysis of this data increased the city’s ability to immediately act, protecting the city against 100 million cybersecurity threat attempts.

AWS Advanced Partner Databricks helped the US Citizenship and Immigration Services, moving 240B data rows to the AWS Cloud in one week. With cloud storage driving down the cost, they no longer needed to decide on limited data to keep, so they increased data sources from 30 to 75. Today, over 7,000 data analysts and scientists leverage this data. Using analytics and ML models, they streamlined application and petition processing and improved query speed by 24 times. They use predictive analysis to plan for how many applications can they expect in future years and the probability of someone not showing up to an appointment.

Getting started

If you are a customer looking for help with a data-led migration, find an AWS Partner here. Learn how to make data a strategic asset in our no-cost ebook.

If you are an AWS Partner and want to join us in helping customers derive value from their data, check out our Migration Acceleration Program (MAP) and our Data and Analytics Competency.

Check out my session from the AWS Public Sector Summit Online to learn more on data-led migrations—now available on demand.