AWS for M&E Blog
How the NFL uses generative AI from AWS to streamline media asset search
Behind every gravity-defying catch and over-the-top tackle, a team of engineers and data scientists work tirelessly to bring National Football League (NLF) fans closer to the on-field action. One of the latest NFL analytics implementations is a sophisticated generative AI system built on Amazon Web Services (AWS). Its creation and deployment is one of the most complex engineering accomplishments in sports media to streamline query and search of media assets to exacting specifications.
How does the NFL take more than a century’s worth of football history, millions of video clips, and an ever-growing archive of statistics as inputs to a system that can respond to conversational queries about this data in mere seconds? This is the puzzle that NFL’s technical team, in collaboration with AWS, set out to solve. From developing advanced machine learning models to creating intuitive user interfaces, the journey to build a system to provide this capability was as arduous and demanding as any fourth-quarter drive. In this blog post, we break down the X’s and O’s of this engineering achievement, demonstrating how the NFL tackles big data challenges head-on.
The challenge: Taming a century of content
The NFL’s journey into a generative AI-powered content management didn’t start on a whim. It was born of necessity, driven by the sheer volume of data the league accumulated over its storied 100-year history. Imagine navigating a digital labyrinth containing millions of video clips, countless audio snippets, an ocean of statistics, and an ever-growing collection of fan-generated social media content. This is the reality of the NFL’s media asset management (MAM) system—a treasure trove of content that is both a goldmine and a challenge for content creators.
The NFL has amassed an unimaginable volume of facts, stats, game day coverage, athlete press interviews, and media assets over the last 100 years, including millions of audio and video clips, as well as still images. Any play of substance merits capture from various viewing angles, such that the NFL’s impressive asset library continues to grow rapidly. It even includes social media clips generated by fans. If roughly half of a game’s plays are consequential, that’s around 75 plays per game, and the average NFL season encompasses 272 games. This means the NFL feeds more than 20,000 plays captured from multiple angles into its library each season.
Matt Swensson, SVP Product and Technology at the NFL, paints a vivid picture of the challenge: “Until now, our teams needed to perform extensive pre-research to find the media assets they needed. It was like trying to find a specific play in a hundred years’ worth of game footage—which is time-consuming and often challenging.”
The ability to search existing assets isn’t just about convenience. In the fast-paced world of sports media, where fan engagement is measured in seconds and content must be as fresh as the latest touchdown, efficiency is everything. The NFL recognized that its traditional approach to content management inhibited its ability to fully capitalizing on its vast media resources.
Swensson explained, “We’ve focused on putting metadata on our media assets to improve searchability, but personnel makes use of the MAM in a traditional sense, and until now, they’ve needed to conduct pre-research to gather breadcrumbs that lead them to the assets they wanted. Our goal is to eliminate all that extra legwork by giving users the ability to make natural language queries that instantly generate highly relevant media asset results.”
Beyond the stats: Generative AI as a storytelling tool
At its core, the generative AI system the NFL developed leverages various large language models (LLM) through Amazon Bedrock, an AWS service that provides a way to build and scale generative AI applications using foundation models. This AI layer sits atop the NFL’s Next Gen Stats (NGS) database, creating a powerful synergy between raw data and intelligent search capabilities.
But what does this mean in practice? Imagine a system that understands and responds to queries as complex and nuanced as the game of football itself. Want to find the number and type of touchdowns quarterback Josh Allen creates when the Buffalo Bills are trailing in a game? Or perhaps understand the Baltimore Ravens’ blitzes against play action pass plays? What once would have taken a team of researchers hours or even days to compile can now be assembled in minutes.
The system’s natural language processing capabilities are a game-changer. Instead of navigating complex database queries or checkbox-laden search interfaces, users can simply ask for what they need in plain English. This helps them find clips that fit specific parameters so they can create content, such as highlight packages for games, narrative storytelling elements, and content packages for NFL digital or socials teams. It’s like having a seasoned sports researcher at your fingertips, one with perfect recall and lightning-fast response times.
Performance metrics: Speed, accuracy, and user experience
In the world of live sports, where every second counts, the performance of an AI system like the one developed by the NFL is crucial. The league set ambitious goals for its new tool, and early results suggest they’ve hit the mark.
Speed of search results was a top priority. “We knew that results needed to start populating within five seconds to give users confidence it was working,” Swensson explains. Rapid response time is critical for maintaining workflow momentum, especially in high-pressure situations like live-game production.
Accuracy was another key metric. The NFL established a baseline goal of 90% accuracy for search results—a high bar given the complexity, variety, and nuance of football data. Impressively, initial tests exceeded this target, showcasing the power of the underlying AI model.
But perhaps the most significant improvement is evident in the user experience. The system doesn’t return a flood of disconnected data points. Instead, it provides consolidated asset links, offering one entry point per play that opens up related assets in the MAM. This streamlined approach prevents overwhelming users with multiple links for every camera angle or audio feed associated with a single play. In addition, the system allows for real-time delivery of results as they are generated. This creates a dynamic, interactive experience that feels more like a conversation with an expert than a traditional database query
From stats to stories: A playbook for the future
While efficiency gains are impressive with this newly engineered approach, the true potential of this AI system lies in its ability to transform raw data into compelling narratives. Mike Band, Senior Manager of Research and Analytics for NFL Next Gen Stats, sees this as just the beginning: “The possible use cases are much greater than the original objective. This tool opens up new opportunities for research and content generation we are just beginning to explore.”
Consider the implications for storytelling in sports media. AI-driven analysis can uncover patterns and insights that can inform even the most experienced human analysts. Perhaps it identifies a subtle correlation between weather conditions and certain play calls, or tracks the evolution of a quarterback’s decision-making over multiple seasons. These insights can fuel richer, more nuanced storytelling that deepens fan engagement and understanding of the game.
“For any given play, we have content from 15 different sources, such as network camera ISOs, radio calls, and social media clips, so we didn’t want query responses pulling discrete links for every asset. Instead, a chatbot provides a summarized response, then offers one link per play that opens up the MAM and lists all the related assets,” noted Swensson. “Teams without access to the MAM also benefit in playing videos and can use the chatbot for research.”
The NFL is already looking ahead to expand the system’s capabilities. Future iterations could incorporate access to news stories, player biographies, and historical media guides. This would transform the application of generative AI from a sophisticated search tool into a comprehensive “NFL expert” capable of providing context and insights that span the entire history and culture of the sport.
Mapping a generative AI future for media search
By harnessing the potential of generative AI, the NFL has revolutionized its asset searchability and consolidated a cumbersome, multi-step process into the simple drafting of a natural language prompt.
For now, the system supports specific NGS-related queries, but Swensson envisions much broader applications. He shared, “We could link to wire stories to incorporate the latest news, such as if a player is injured or cut, and make biographical information available. We probably have 1500 media guides, which are hundreds of pages filled with everything from facts and stats to personal anecdotes. We can train an LLM on all these data sources so that it’s an NFL expert that can provide knowledge outside of Xs and Os.”
Band concluded, “It has been great to have the support of AWS to guide us through the entirely novel world of LLMs, and our embrace of generative AI signals the NFL’s commitment to being tech-forward. There’s no telling what we can do from a research perspective or from a text-based content generation perspective, so this is really just the start of myriad possibilities.”
Learn more about how the NFL is using generative AI on and off the field.