AWS Partner Story: InfoTrack

InfoTrack, an award-winning Australian technology company, provides intelligent search and automated workflows for companies across the professional services industry.

InfoTrack, an award-winning Australian technology company, provides intelligent search and automated workflows for companies across the professional services industry. The company’s SaaS solutions can be seamlessly integrated into third-party practice management, document management, and accounting systems to help customers quickly find information pertaining to property, company, personal, and other search data that use built-in electronic signature capabilities to help further streamline workflows.

Founded in 2001, InfoTrack functioned primarily as an aggregator of information for several years, pulling together data from government authorities and other sources into a single, searchable repository. This was the company’s “bread and butter” for some time, and InfoTrack did it well, adding new data sources and serving new industries as the company rapidly grew.

At the start of 2017, the company’s board of directors decided to create an Innovation Team. Its mission: explore the development of new products that combine existing data sources with entirely new ones—and do more with that combined data than simple aggregation to deliver new customer value. “Our mission is to envision and experiment to see what works and what doesn’t,” says Joel Cunningham, General Manager of Innovation at InfoTrack. “It’s perfectly OK to fail, but if that’s going to happen, we want to know it fast so that we can move on and try something else.”

Richard Morwood, manager of big data, was tasked with building a technology platform that could support the board’s vision. He knew from prior experience that he would use Amazon Web Services (AWS), which would give the company all the core services needed for a scalable big data platform in the cloud, and set out to find the right tools for bringing it all together.

“I knew we would have many different data sources, which we had to combine into a single source of the truth—and AWS gave us all of that,” says Morwood. “However, we still needed tools for quickly and easily working with all of that data, in a way that would let us avoid writing a lot of time-consuming and error-prone SQL code by hand. Basically, we wanted the ability to prototype a solution in an afternoon.”

Morwood selected Matillion ETL for Amazon Redshift, a solution from Matillion, an Advanced Technology Partner of the Amazon Partner Network (APN). Delivered as an Amazon Machine Image (AMI) via the AWS Marketplace, Matillion ETL for Amazon Redshift unlocks the power of the fully managed Amazon Redshift data warehouse service by simplifying data loading, transformation, and orchestration through a full-featured, browser-based, graphical job development environment.

“With Matillion ETL for Amazon Redshift, it only takes a few clicks to generate the necessary SQL code, enabling us to quickly try out an idea and just as quickly change direction if needed,” says Morwood. “And not only is it faster, it’s also more accurate than writing SQL code by hand.”

Since adopting AWS and Matillion, InfoTrack has used the combined big data platform to introduce two products. Following is an architectural diagram for the first project, a United States Property Search service, which enables users to quickly find a property, view basic information in it, and, if desired, order a report that provides additional detail—such as the property’s owner or whether there are any liens against it.

Matillion solution architecture

Here’s how the solution works:

● Daily updates from an external data provider are captured in Amazon Simple Storage Service (Amazon S3), after which the “daily delta” is captured in Amazon Redshift staging tables.

● Matillion ETL for Amazon Redshift orchestrates data clean-up and correlates property ownership to addresses, then loads that information into the main data warehouse running on Amazon Redshift.

● Information in the main data warehouse is pushed to two places: Amazon OpenSearch Service, which provides an indexed data catalog and powers the search box on the public website; and Amazon Aurora, which stores all the details for each property.

● When a user selects a property via the search box, Amazon ES retrieves the detailed property data from Amazon Aurora via Amazon API Gateway, which in turn calls an AWS Lambda function to access the database.

Continuous integration/continuous delivery (CI/CD) is achieved through the use of AWS CodePipeline and AWS CodeBuild, with deployment handled by AWS CloudFormation. “Our current United States Property Search solution matches address-to-owner,” says Morwood. “We’ll soon be delivering additional products from the same data set, using the same architecture. 

Powered by AWS and Matillion, InfoTrack’s new big data platform in the cloud enables exactly what the company’s board of directors envisioned when it formed the company’s Innovation Team: the ability to ask “What if we did this…?”, see how it works, and if it delivers the envisioned new customer value, quickly productize it. Specific benefits of the company’s choice of technology include:

● Rapid time-to-value. In the short time since it adopted AWS and Matillion, InfoTrack has already delivered multiple new products. “We started building-out our Innovation Team in late February 2017, from a standing start,” says Morwood. “And we’ve already accomplished a lot. A big part of that is because, with AWS, we haven’t had to worry about servers, patches, software licensing, and all other ‘hassles’ associated with building-out our own physical infrastructure. Put another way, we’re able to focus on what the customer needs and sees, not how we’ll deliver it.”

● Reusable architecture. InfoTrack’s data platform architecture is reusable, providing support for new products with little more than another front-end database. “With all our data sources related and linked together in Amazon Redshift, as orchestrated by Matillion, we have a single source of the truth—from which we can push subsets of that data to customer-facing databases to quickly deliver new products,” says Morwood. “It’s an incredibly agile and powerful architecture. And with deployment driven by AWS CloudFormation, we can duplicate it anywhere in the world.”

● Strong scalability and performance. With its data platform in the cloud, InfoTrack doesn’t need to worry about scalability and performance. “We don’t always know how much data we’ll need to handle, but it’s not something that concerns us,” says Chao Mu, data developer on the Innovation Team at InfoTrack. “Amazon Redshift is really, really fast. And with it, we can choose a starting point in terms of compute resources and see how it works, easily experimenting with different sized clusters and instances until we find the right fit.”

● Powerful ETL tools. Matillion ETL for Amazon Redshift gives InfoTrack a comprehensive tool-set for putting the power of Amazon Redshift to use. With it, the company can easily load data from a broad range of sources, transform it at impressive speed; fully orchestrate data movement; integrate with other AWS services; and maintain comprehensive version control. And because it runs as an AMI, Matillion ETL for Amazon Redshift can be scaled as easily as the underlying Amazon Redshift cluster.

Looking back at all that InfoTrack has accomplished since it began assembling its Innovation Team, Morwood is confident that he made the right decision choosing AWS and Matillion. “We’ve already developed five new products; two are already in the market, and another three are yet-to-be-announced,” he says. “AWS is a powerful cloud platform, and Matillion ETL for Amazon Redshift gives us the tools we need to put that platform to proper use. I’ve been just as impressed by the support we’ve received; the way Matillion always treats us as if we’re their number-one customer is amazing.”

To learn more about AWS Data Warehouse solutions, visit our Amazon Redshift page.