Verisk: Migrating to Amazon Aurora with AWS Data Lab Case Study
As a result of our work in the AWS Data Lab, the team saw a concrete shift in our progress from theoretical to actual, and we now understand precisely what needs to be accomplished to migrate fully into production. The Data Lab experience enabled us to get the right AWS and Verisk leaders in the room to map out and mobilize our migration in a very real and material way.”
To scale solutions quickly and achieve greater resilience against points of failure, Verisk made a strategic decision in 2017 to mandate the migration of all corporate-wide IT systems to the cloud by 2022. This impacted legacy databases across multiple business units, which Verisk chose to migrate to Amazon Web Services (AWS) for increased agility and lower costs.
Like most enterprises undertaking a mass migration, Verisk’s IT teams had to navigate design, architectural, and implementation challenges, including concerns about potential migration risk due to extensive use of complex data types like large objects (LOB) and geospatial data, large volumes of data, and complex procedures and schemas that were developed over 20+ years.
Working with AWS Data Lab
Verisk chose to collaborate with AWS Data Lab for expert guidance on these migration challenges. In April 2019, Verisk participated in its first AWS Data Lab to focus on the migration of two databases that included tables that were partitioned with billions of rows and required complex transformations for LOB datatypes. Using AWS Database Migration Service (DMS) and AWS Schema Conversion Tool (SCT), AWS Data Lab helped the Verisk team architect and prove out a migration path to move data from two on-premises databases to Amazon Aurora PostgreSQL, capture change data, and integrate its applications with the new target databases. Verisk chose Amazon Aurora PostgreSQL because of its high availability, backup and restore capabilities, cluster cache management, fast cloning, and replication functionality. The Verisk team finished the week with greater technical skills and a proven, repeatable process it could use to migrate Verisk’s remaining data to AWS.
Several months later, a second and third business unit at Verisk uncovered additional complex migration challenges. In December 2019, Verisk chose to re-engage AWS Data Lab to migrate two of its most complex schemas from its legacy database to Amazon Aurora PostgreSQL using AWS SCT, refactor and modernize its existing PL/SQL application code to Java, and refactor 14 legacy procedures into a single procedure. “Quality expertise was provided for us to break through processes vastly quicker than self-research. Many tips were provided around best practices and possibilities that we would have not realized we needed to know for months. The AWS Data Lab format was very good, allowing us to take use cases from our office to work on and then provided artifacts to bring back as working models,” shared a Verisk builder who participated in the AWS Data Lab.
Soon to follow in January 2020, Verisk’s third team joined the AWS Data Lab for guidance migrating from its legacy spatial database to AWS. The AWS Data Lab helped Verisk choose the right AWS database option based on its business requirements, which was Amazon Aurora PostgreSQL. In only four days, the Verisk team used AWS DMS and SCT to successfully convert a subset of its spatial schema to PostgreSQL with PostGIS, migrate 85 spatial and 135 non-spatial tables to PostgreSQL, and run initial functionality tests on its downstream application after integrating it with the new target database. The AWS Data Lab provided Verisk with access to an AWS DMS feature for migrating spatial data to PostgreSQL, which at the time was not publically available. The feature was released later in Q1 2020. Another Verisk builder noted, “As a Database Administrator at Verisk working on the data migration, I am miles ahead of where I was prior to working with the AWS Data Lab. I have more confidence in being able to successfully migrate our legacy database to Aurora PostgreSQL and have a better understanding of what products are available to us. I couldn't have asked for a better experience.”
In addition to the technical work achieved in the AWS Data Lab, Verisk came away with an increasingly focused migration strategy, a deepened understanding of how to execute migrations to AWS databases, and best practices for database administration and operating PostgreSQL databases in production, such as how to create database parameter groups, define a backup and failover strategy, monitor performance and configure alarms using Amazon CloudWatch, vacuum, clone, and audit.
“As a result of our work in the AWS Data Lab, the team saw a concrete shift in our progress from theoretical to actual, and we now understand precisely what needs to be accomplished to migrate fully into production,” noted Tim Coyle, CIO, ISO at Verisk. “The Data Lab experience enabled us to get the right AWS and Verisk leaders in the room to map out and mobilize our migration in a very real and material way.”
Verisk is a leading data analytics provider serving customers in insurance, energy and specialized markets, and financial services. Using advanced technologies to collect and analyze billions of records, Verisk draws on unique data assets and deep domain expertise to provide first-to-market innovations integrated into customer workflows. It offers predictive analytics and decision support solutions to customers in rating, underwriting, claims, catastrophe and weather risk, global risk analytics, natural resources intelligence, economic forecasting, and many other fields.
About AWS Data Lab
AWS Data Lab offers accelerated, joint engineering engagements between customers and AWS technical resources to create tangible deliverables that accelerate data and analytics modernization initiatives. During the lab, AWS Data Lab Solutions Architects and AWS service experts support the customer by providing prescriptive architectural guidance, sharing best practices, and removing technical roadblocks. Customers leave the engagement with a prototype that is custom fit to their needs, a path to production, deeper knowledge of AWS Databases, Analytics, and Machine Learning services, and new relationships with AWS service experts.
AWS Services Used
AWS Database Migration Service
AWS Database Migration Service (DMS) helps you migrate databases to AWS quickly and securely.
AWS Schema Conversion Tool
The AWS Schema Conversion Tool (SCT) makes heterogeneous database migrations predictable by automatically converting the source database schema and a majority of the database code objects to a format compatible with the target database
Amazon Aurora is a MySQL and PostgreSQL-compatible relational database built for the cloud.
Amazon CloudWatch is a monitoring and observability service built for DevOps engineers, developers, site reliability engineers (SREs), and IT managers.
Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.