How Musicians Use AWS to Go from Big Data to Their Big Break

By Rob Koury, Managing Partner at Vertical Trail

With fans around the globe using digital technologies to consume music at an unprecedented rate, thousands of emerging artists are now reliant upon music streaming as their main source of exposure. New pathways are opening for musicians to reach their fans every day, but the royalties from streaming services alone are not typically sufficient to support their career.

Despite the advances in music-sharing technology, touring and live shows remain staples for emerging artists. So how do artists and promoters align the reach of streaming services and online ticketing with fans’ physical locations to better plan, promote, and manage live shows?

To answer this question, Vertical Trail partnered with Gigable to create a music platform and app that uses curated playlists, geo-based concert discovery, and online ticketing to close the gap between artists and their fans—all while building a formidable big data asset.

Vertical Trail is an AWS Partner Network (APN) Advanced Consulting Partner with AWS Data & Analtyics Competency and Amazon EMR Service Delivery designation. Amazon EMR provides a managed Hadoop framework that makes it easy, fast, and cost-effective to process vast amounts of data across dynamically scalable Amazon Elastic Compute Cloud (Amazon EC2) instances.

How Gigable Works

Gigable Music Discovery helps music fans find their next favorite band and see them in concert. Their curators scour thousands of playlists and outlets to find cream-of-the-crop emerging music, and listeners can easily uncover new artists and purchase concert tickets.

Via direct relationships, these artists partner with Gigable to start building “target” regions. Listeners using the app can save their favorite songs and set reminders for upcoming shows. This information is shared with artists and promoters, who leverage those analytics to learn where their fan bases are concentrated.

Over time, this geodata, along with “superfan” data (i.e. those with the highest listener hours), presents intelligent fan clustering data that shows the full picture of where fans are located, as well as their purchasing intent with respect to shows.

Figure 1 – The Gigable music app allows listeners to easily stream curated playlists and find nearby shows.

How We Built It

Vertical Trail consultants built a strategy and design for a scalable analytics warehouse capable of handling an accelerated amount of streaming data. This high-volume and rapidly
generated data includes full and partial song plays, session duration, navigation, and other factors contributing to the big picture of mobile analytics.

The application of back-end API calls and other analytics tools and resources resulted in a multidimensional dashboard providing a highly accurate picture of the listener lifecycle.

Leveraging unstructured listener event data from the app and implementing a customized Smartbots.ai solution allows Gigable to ask questions of the data. The results are used to inform
artists’ management companies of hyper-local information relevant to successful tour planning.

Sample queries can include:

• How many listeners have saved show XYZ?
• How many unique listeners have favored song ABC?
• What are the top 10 songs in Illinois?
• What is the most favored song in Illinois over the past month?

The deployment of this simple Smartbots interface allows more efficient, rapid delivery of insights to artists’ managers—without the need to develop elaborate visualizations.

New Technology

Vertical Trail provided a growth strategy that enabled comprehensive data collection as app usage accelerated, via the use of key technologies including:

• Amazon Simple Storage Service (Amazon S3) and Amazon EC2, which allowed for lower cost of deployment and high elasticity of the solution.
• Apache Hadoop, which played a primary role in establishing a data lake capable of growing with the company.
• Hive, which enabled different types of source data to be collected and stored in a uniform manner for efficient analysis.
• Smartbots.ai, built on the AWS Lex platform, enabled ad-hoc access to critical data points via simple chat commands, and sped up access to insights that would otherwise take full development cycles to produce.
• Amazon EMR, with a configuration comprised of:
  • Hadoop 2.8.3 with Ganglia 3.7.2, Hive 2.3.2, Hue 4.1.0, Mahout 0.13.0, Pig 0.17.0, and Tez 0.8.4
  • An instance type m3.xlarge (i.e. three total instances, one master, and two core)
  • Configuration of Identity and Access Management (IAM) credentials for the Gigable Virtual Private Cloud (VPC) and proper credentialing on the Amazon EMR cluster
  • Amazon CloudTrail for the specific bucket that holds the song files so that when a song file is requested via API the object event is logged to EMR; these events are then parsed and
    used by Hive and Hue to construct tables
  • Custom dashboard showing the daily/weekly/monthly activity for each song that can be used to combine data and find the most popular songs

Reference Architecture

Figure 2 – The back-end architecture developed by Vertical Trail and Gigable highlights the application of big data components and capacity for growth.

Customer Success

One of the most influential talent buyer/venues in the Midwest says, “We’re very optimistic about the ad hoc data we’re seeing from Gigable. By testing their Smartbots interface, we are quickly able to hone in on fan behavior and get immediate insights on key stats that will help us plan and market shows.”

The bot is being trained to understand the context of touring and live shows. Over time, the bot will be able to provide predictive analytics that flag an emerging artist based on listener velocity and help talent buyers prioritize their show planning.

The end result of our efforts with Gigable was the creation of an application with immense benefits for both fans and artists. On the listeners’ side, exposure to high-quality emerging music enhances their discovery experience, and their usage data assists in creating real-world opportunities to see their favorite artists live.

Synergistically, artists and promoters can accelerate their career development and growth with access to targeted insights that would be otherwise lost in the shuffle of streaming data.

Next Steps

Learn more about Vertical Trail and how our data-drive solutions can help transform your business.

The content and opinions in this blog are those of the third party author and AWS is not responsible for the content or accuracy of this post.

.

Vertical Trail – APN Partner Spotlight

Vertical Trail is an APN Competency Partner. They provide data architecture and cloud consulting services, employing unique and proven framework to ensure every project is aligned with customers’ business objectives and delivered with an eye to maximum satisfaction.

Contact Vertical Trail | Practice Overview | Customer Success

*Already worked with Vertical Trail? Rate this Partner

*To review an APN Partner, you must be an AWS customer that has worked with them directly on a project.