AWS Partner Network (APN) Blog
Choosing the Right Tool: Selecting Machine Learning or Generative AI for Optimal Results
By Joaquin Lippincott, CEO – Metal Toad
By Hector Leano, Head of Marketing – Metal Toad
By Karunakar Kotha, Senior Partner Solutions Architect – AWS
![]() |
Metal Toad |
![]() |
While the terms “Generative AI” and “Machine Learning” are sometimes used interchangeably, they are distinct technologies serving different purposes for different scenarios. Traditional machine learning (ML) is focused on outputting predictions based on patterns found in training data. Think of something like computer vision, where a model scores the likelihood of a given image being a horse based on all the past photos tagged as horses in its training data set.
Generative AI (GenAI) on the other hand is a relatively new subfield of ML focused on synthesizing, or generating, new content such as text, code, or images based on text prompts Large Language Models (LLMs) like GPT-4, Claude, and Llama 2 are prime examples of GenAI systems that can process and generate human-like text. In this case, you are asking the model to generate an image of a horse, which it does based on all the horse pictures it has in its training data.
While generative AI is powerful because it can create new content instead of just evaluating it, it might not always be the best tool for your needs.
Regular machine learning works best when you have lots of good, labeled examples to train with and they match the real-world situations where you’ll use them. Think of it like this, if you have clear examples of what you want the system to do, and these examples are similar to your actual needs, traditional machine learning is often the better choice.
Case Study: Proof of Concept for Advanced AI Solutions with AlertWest
Recently, Metal Toad, an AWS Advanced Consulting Partner, collaborated with AlertWest, a public safety initiative, to execute a proof of concept (POC) project aimed at advancing AI capabilities. The POC was designed to explore a novel machine learning approach tailored to the unique needs of AlertWest’s mission of public safety monitoring and response.
Leveraging AWS services, including Amazon SageMaker, PySpark, and PyTorch, Metal Toad developed and tested a custom AI model. This model introduced a new analytical framework capable of handling complex datasets, providing insights beyond conventional solutions. The POC emphasized feasibility, accuracy, and alignment with operational goals, offering a foundation for future applications while validating the potential for scaling the approach.
Through close collaboration with AlertWest, Metal Toad ensured the project not only delivered technical breakthroughs but also addressed the practical challenges of deploying AI in high-stakes environments.
Figure 1. Operators Manually Monitoring Footage to Detect Wildfires
Generative AI: For When Context Matters
But for many scenarios, you often won’t have well-labeled data representative of all possible inputs you might see in production. In those cases, GenAI can help with its ability to use the surrounding context when generating an answer.
Metal Toad faced a similar challenge when working with GeniusVets. The goal was to scan thousands of phone calls coming in to veterinary hospitals and identify any calls with negative client interactions in order to improve staff training. However, with thousands of hours of call transcripts generated every day, the customer service team was not able to adequately analyze the data.
They tried using ML-based sentiment analysis tools, but those failed because calls relating to sickness or injuries (which is most of what a veterinary hospital deals with) tended to be automatically flagged by sentiment analysis as “negative”. Hiring someone to manually review these call transcripts and tag the recordings would be costly and still potentially prone to error.
Properly categorizing call interactions requires an understanding of the context of the whole conversation, and not just individual data points. Metal Toad identified that this is a good use case for generative AI, which is able to look at how the words in the transcript relate to each other and the conversation as whole, and not just in isolation.
Utilizing AWS services including Amazon Bedrock and Amazon Transcribe, Metal Toad created a system that more accurately identifies calls with a negative sentiment that need further review. This enables business users, like call center managers, to quickly identify trends and areas for improvement.
Figure 2. Reference Architecture
Working within GeniusVet’s customer service platform (Twilio), Metal Toad converted speech to using Amazon Transcribe. The resulting transcripts were analyzed using LLMs on Amazon Bedrock in order to query for specific behaviors such as flagging calls where customers do not have a vet appointment.
Studying thousands of calls through a combination of software and human evaluation, the teams worked side-by-side to train the AI models to spot calls that needed attention. By focusing on the top 10% of calls that need attention, managers have become 10X more efficient in their call reviews, leading to faster growth in revenue for GeniusVets and their clients.
Deciding Between Traditional ML and Generative AI
Customer obsession is a key tenet for AWS and Metal Toad. We believe that the solutions should be as simple as possible because being able to effectively operate and enhance the platform over time is important.
When starting a generative AI, machine learning, or data analytics engagement, Metal Toad likes to conduct a strategy session with customer stakeholders to better understand the business objectives and data landscape.
Figure 3. The Metal Toad Process for building genAI and ML applications for firms of all sizes.
From there we select a use case to build a proof of concept (POC) that provides not just technical validation (can it be done) but also business validation (is it worth it). When you see stats like 90% of genAI POCs fail to move into production, this is often because the business use case was an afterthought in the POC design process. Part of working with a tech consultancy like Metal Toad is our experience understanding what it takes to take an idea from POC to implementation, particularly by understanding the key questions you need to answer when designing a POC with an eye towards building a real business application:
Questions Answered by a Properly Designed POC
Technical Validation (can we do it?)
-
- Do we have the right data? If not, can we get it?
- Do we know how to get the data from the source to where the inferencing/decisioning happens?
- Can we deliver the output to downstream applications and users that need it?
- Do we have the right internal resources to maintain this application (such as data engineers and data scientists)?
Business Validation (is it worth doing?)
-
- Is the output of the model better than whatever we use now?
- Can the business actually use that output?
- Does this provide a rough idea of what the costs will be to run this application?
- Are these costs less than the expected incremental revenues/cost savings?
For small dev teams in startups and small to midsize businesses (SMBs), integrating AI and ML into production use cases often requires specialized resources in data engineering and data science. Metal Toad is the AWS Advanced Tier partner with over 50 AWS certifications focused specifically on serving startups and SMBs.
As a result, even small dev teams can access AWS-certified solution architects, data scientists, engineers, and front-end developers to build end-to-end AI/ML applications or to cover gaps to their own resources.
Conclusion
Metal Toad are experts in applying Machine Learning and Generative AI to solve business challenges.
POCs rapidly validate technical feasibility and reduce implementation risks. At Metal Toad, we excel in fast prototyping and turning concepts into practical solutions through our proven agile methodology and deep technical expertise.
Metal Toad – AWS Partner Spotlight
Metal Toad is an AWS Advanced Consulting Partner and Managed Cloud Services Provider who specializes in utilizing the latest AWS tools, like artificial intelligence to solve business problems. Whether migrating away from VMware or deploying the latest generative AI model using Amazon Bedrock, Metal Toad helps your business get things done.