HST Pathways Builds a Near Real-Time Data Warehouse for Rapid Analytics in Healthcare Management

aws-data-lab
HST Pathways

Challenge

Achieving near real-time latency with disparate data sources: In one year, HST Pathways merged with four companies with different tech stacks and products. As a result, data for their healthcare provider clients was spread across hundreds of databases, both on-premises and across three different cloud providers, making it difficult and time-consuming to get a holistic view of their data and leverage it for decision-making.

HST Pathways wanted a solution that could simplify data aggregation and storage while giving them the flexibility and performance needed to run analytics on large, multi-tenant data sets. As the healthcare industry is further empowered by new technology, HST Pathways foresees the growing demands from sophisticated customers for near real-time data analytics. Therefore, one performance requirement is to update critical data every 15 minutes in their data warehouse.

Solution

Building a data warehouse for large-scale analytics: HST Pathways collaborated with the AWS Data Lab to develop a centralized data lake from which they could build data warehouses custom-fit to the needs of their customers: individual data warehouses for their large enterprise customers and a combined data warehouse to run analytics for their customers with smaller data sets. This solution includes a low-latency pipeline that's capable of ingesting data from 100+ on-premises SQL Server tables into Amazon Redshift, a fully managed, scalable cloud data warehouse on AWS.

Using AWS Database Migration Service (DMS), HST Pathways built a near real-time solution that streams data into an Amazon Simple Storage Service (S3) data lake. Lastly, the team created an Amazon Redshift Spectrum pipeline that applies incremental updates to their Redshift aggregate tables every 15 mins to support near real-time data analytics on clinic-critical data. See architecture diagram below.

Benefit

Speed, scale, and a workable data foundation: HST Pathways deployed their data warehouse solution to production within only six weeks of working with the Data Lab. The Build Lab created an environment where HST Pathways could design and build a prototype of a near real-time data warehouse, a centralized data lake, and a scalable data streaming service in only four days.

The data lake built during the lab will serve as HST Pathways’ central hub for long-term data consumption and analytics for a broad spectrum of healthcare customers.

HST Pathways also had time during their Build Lab to test out several prototype alternatives with their dedicated Data Lab Architect, helping their engineering team upskill on best practices for near-real time streaming and search that will prepare them for tackling future analytics use cases for the organization.

“Working with the AWS Data Lab was intensive and productive. AWS Data Lab Architects helped us work backward from our business needs and try different potential options for our data warehouse project. After the four day Build Lab, we were confident that AWS DMS and Amazon Redshift are the best technologies for our needs, and we were able to quickly launch to production within weeks.”

Xiaodong Chen, Manager of Software Engineering, HST Pathways.

  • AWS Services Used
  • About HST Pathways
  • HST Pathways is an innovative software technology company enabling healthcare providers to create better patient outcomes through a suite of digital solutions for practice management, electronic charting, and case coordination.

  • About AWS Data Lab
  • AWS Data Lab offers accelerated, joint engineering engagements between customers and AWS technical resources to create tangible deliverables that accelerate business modernization initiatives on AWS. During the lab, AWS Data Lab Solutions Architects and AWS service experts support the customer by providing prescriptive architectural guidance, sharing best practices, and removing technical roadblocks. Customers leave the engagement with a prototype that is custom fit to their needs, a path to production, deeper knowledge of AWS services, and new relationships with AWS service experts.

Architecture Diagram

HST Pathways Architecture Diagram

Get Started

Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.