AWS Partner Network (APN) Blog
Avahi Migrates MasterWorks’ Machine Learning App to AWS to Lower Cost and Speed Up Data Modeling
By Nirav Shah, Principal Solutions Architect – AWS
By Jack Singh, CEO – Avahi Technologies
 |
| Avahi Technologies |
 |
The need to reduce costs is often a primary driver when IT leaders consider migrating from a software-as-a-service (SaaS) application to a new hosting provider. Even if applications that customers rely on perform sufficiently, the business needs to make sure they generate enough margin so the applications produce a profit.
Migrating to a new hosting provider to save costs also presents an opportunity to fine-tune application performance and perhaps even the DevOps processes supporting the applications.
This was the case for MasterWorks, which is based near Seattle and helps move the hearts and minds of people to act for Christian ministries across America. For more than 30 years, the company has partnered with an exclusive group of ministries to build passionate audiences and raise money for their missions by providing solutions that help engage constituents through creative marketing campaigns and best-of-breed technologies.
In this post, we present how Avahi Technologies and Amazon Web Service (AWS) collaborated to help MasterWorks migrate an application that uses machine learning (ML) models from a SaaS provider to AWS.
Avahi was well-suited for this project with its ability to help customers leverage a cloud-first strategy. Avahi is an AWS Advanced Tier Services Partner and AWS Marketplace Seller that offers a team of certified cloud, data, and software engineering experts who excel at architecting and operating secure and automated solutions built on AWS.
Identifying ML Experts to Migrate Application in 8 Weeks
A key application that MasterWorks provides to clients uses machine learning models to measure the propensity that an organization’s members will donate to a particular fundraising campaign.
Clients use intelligence produced by the audience selection tool to narrow down their member databases and focus marketing efforts on segments most likely to give money and other gifts. The service essentially allows clients to reduce the expenses of fundraising campaigns while increasing net revenues.
“We relied on a hosted SaaS application that was working well enough, but the provider’s licensing model made it difficult to operate the application profitably as we expanded our client base,” said Milo McDowell, Sr. Vice President of Operations at Masterworks. “As our hosting contract began to approach the annual renewal date, we decided to look for a lower-cost hosting provider.”
To reduce the costs for hosting the ML application, McDowell first turned to AWS, the cloud provider MasterWorks has relied on for other application workloads for more than 10 years.
McDowell realized AWS could offer a cost model that would reduce the expense of hosting the application, and that Amazon SageMaker provides a platform to streamline processes to create, train, and deploy machine learning models in the cloud. McDowell also realized he would need help in migrating the application to AWS.
“We did not have the capacity on our internal staff to handle such a complex project,” McDowell explained. “We also did not have anyone on our team with relevant SageMaker experience.” Thus, finding a SageMaker expert became the prime need.
MasterWorks also faced another challenge: “We had about eight weeks to complete the migration before our hosting provider contract would renew,” McDowell said. “We needed to make sure we could complete the project relatively quickly.”
Avahi Instills Confidence to Migrate 200 Models on Time
Key challenges of the project included:
- Reduce SaaS machine learning platform costs.
- Avoid diverting the internal IT team from core responsibilities.
- Identify expertise to migrate to a new ML platform in eight weeks.
Thanks to a strong working relationship with AWS, an answer for McDowell was right around the corner as AWS referred MasterWorks to Avahi. After the initial meeting to discuss migrating the ML models to Amazon SageMaker, McDowell felt confident Avahi could complete the project within the short timeline.
“The Avahi team dived right in to assess the 200 models used by 35 clients that we needed to replicate,” McDowell said. “They demonstrated their experience with SageMaker and took the time to understand our needs. From there, Avahi developed a plan for how to complete the migration by our deadline while also recommending ways to enhance our machine learning models.”
Given the pending deadline and the confidence Avahi instilled, McDowell did not consider collaborating with any other firms. “From the developer to the project manager and the CEO, Avahi impressed us with their commitment to delivering the project on time. We did not feel like we were taking a risk with Avahi,” he said.
Zero Post-Production Issues
Avahi created a pre-processing workflow (see Figure 1) using the Matillion ETL tool to extract and store data as a .CSV file in Amazon Simple Storage Service (Amazon S3). The workflow also transforms the data model into fixed features as expected by SageMaker to predict target variables and generate results.
Figure 1 – Amazon SageMaker reads input files and infers predictions using hosted models.
Amazon SageMaker provides batch transformation jobs via an API that does not require a dedicated hosted endpoint for a model. With this capability, MasterWorks can run comparisons between two batches of inference results.
The model training pipeline has six model versions for each MasterWorks client that can be used for different types of predicted targets, and the system automatically trains, tunes, and selects the best model for each type of inference to make it readily available on-demand. MasterWorks can easily leverage the input to train models for a new client or retrain newer versions for existing clients.
The inference pipeline is cost-optimized by relying on serverless computing and on-demand batch requests powered by AWS Lambda and AWS Glue. The pipeline automatically generates predicted results when a file is dropped into S3. In addition, the pipeline chooses the latest version of the model for a client and model type if there are multiple versions of the same type of model.
“The project went smoothly, and it was clear the Avahi team worked hard to make it happen,” said McDowell. “As we cut over and went live into production, we needed to adjust how the models run inferences. Avahi quickly helped us determine how to reprocess jobs—their team worked well with our team, and we haven’t had any issues since going into production.”
Costs Reduced 75% as Model Processes Accelerate
By migrating the machine learning model application to AWS, McDowell estimates MasterWorks has reduced hosting and usage costs by approximately 75%. One of the reasons why AWS is less costly than the previous provider is that MasterWorks can use compute resources on-demand. With the previous SaaS platform, MasterWorks was assigned dedicated machines that were always running.
“We also save because the AWS licensing model is different,” added McDowell. “We pay for compute resources in AWS, but we don’t have any user charges to go along with that. The SaaS provider charged a licensing fee for each user, and where our utilization rate falls between 5-10% it was not a good use of our money to spend it on a full-time state-of-the-art machine learning platform.”
In contrast, migrating to AWS allows MasterWorks to run the same data models without paying the hefty overhead of the previous platform. “As we use additional resources to develop new models with SageMaker, there’s no overwhelming pressure that we’re spending too much money on licenses,” McDowell pointed out. “We’re spending dramatically less and can maximize our machine learning efforts.”
The money MasterWorks is no longer spending on the previous machine learning platform can be invested in other initiatives. In addition to the cost savings, McDowell says the processes to train new models and run inferences operate faster in the AWS environment.
“By running new data against a trained model to infer what will happen during fundraising efforts, our engineers can generate more accurate results for clients to measure the propensity that members will donate to various campaigns,” McDowell said.
This capability helps clients evaluate campaign strategies before investing resources. Life is also better for the MasterWorks engineers working in SageMaker, as they can run model training and inferences concurrently while benefitting from the detailed solution documentation provided by Avahi.
“Our engineers refer to the documentation if they have questions relating to the resources, and we can more easily cross-train other engineers on our internal team,” McDowell said. “We now have a structure for building training new models with a detailed process that our engineers are comfortable with.”
Key results included:
- Avoided SaaS renewal fees by completing migration on time.
- Decreased application hosting and license costs by 75%.
- Accelerated model training and inferences processes.
- Provided detailed documentation to assist internal engineers with data model resources.
Collaboration Offers Powerful Combination of Skills
Streamlining the process to build and train data models is vital as MasterWorks goes through model training each time a new client comes on board. “We’re also always working to improve our services, so as we innovate we go through rigorous testing of new ideas and can incorporate them into the models for existing clients,” McDowell said.
In addition to the data modeling project, Avahi is upgrading the code base of an Epiphany business management application to work properly in the modern AWS compute environment. The application is written in the NodeJS language and hosted on AWS Elastic Beanstalk. Previously, Epiphany ran too slow with limited functionality on an outdated version of NodeJS. After Avahi refreshed the application with the latest libraries and new features, the application operated much faster.
“We work with a lot of consultants and partners, and Avahi stands out for their machine learning expertise as well as the effort they put in and the documentation they provide,” said McDowell. “That’s a powerful combination of skills, and it’s rare to see an IT partner deliver on all three the way Avahi does.”
Learn more about Avahi in AWS Marketplace.
.
.
Avahi Technologies – AWS Partner Spotlight
Avahi Technologies is an AWS Advanced Technology Partner that offers a team of certified cloud, data, and software engineering experts who excel at architecting and operating secure and automated solutions built on AWS.