Observability Matters at Brightcove with AWS GameDay

Today, we’re pleased to announce the general availability of the Observability Matters on Amazon Web Services GameDay. AWS GameDay is a gamified learning event that challenges participants to use AWS solutions to solve real-world technical problems in a team-based setting. Unlike traditional workshops, GameDays are open-ended and non-prescriptive to give participants the freedom to explore and think outside the box.

Built with purpose

The pressures to keep systems up and running with minimal disruption are increasing while applications, infrastructure, and services are being deployed in ever-increasing rates. Businesses’ accelerated migration to the cloud is also adding unprecedented complexity to computing environments such as containers, open source, and managed AWS services as customer accounts and infrastructure options increase. IT teams struggle to keep up with a variety of application types, coordinate with stakeholders, gain visibility into siloed data, and stay within budget. Deploying observability solutions at scale then becomes a challenge as each component of a solution differs from another solution. Before, logs and metrics of a physical host like CPU and syslog may have been enough. But now, it has evolved to observing based on the customer journey, and doing so efficiently. This doesn’t mean traditional metrics are irrelevant, but rather, the necessity to observe from the lens of the consumer who steps through the solution across multiple services became increasingly important. This shift in mentality redefines the right telemetry data to debug an issue or large-scale events. To support customers like Brightcove to evaluate their current skillsets and educate customers on observability, we looked for a fun, developer focused solution.

Brightcove Inc. is a leading global provider of cloud services for video which enable its customers to edit, manage, publish and distribute video to Internet-connected devices quickly, easily and in a cost-effective and high-quality manner. Through its streaming technology, Brightcove also helps its customers monetize their video content and gather analytics, metrics and insights to fuel their strategy with a data-backed approach. To support their complex solutions and geo-disperse customers, Brightcove looked to AWS to evolve their observability practices and culture even more.

Brightcove strives to create reliable, scalable, and secure streaming solutions to their global customers. When it comes to AWS observability practices, Brightcove is not new to the concept. Brightcove leverages many of the AWS telemetry data to surface them in the way it matters the most to their customers. Brightcove wanted to focus on building their solutions to proactively alert on failures before their customers notice any degradation, in complex, microservice drive, real-time architecture. And they needed to periodically check in on their team for any new practices and AWS services. So here came the AWS GameDay. Brightcove is the first customer to run Observability Matters and provided valuable feedbacks to the development team to polish this GameDay before the general availability.

“With the rapidly evolving technology landscape, it’s imperative for us to innovate and look at reliability through the lens of observability. We hosted an AWS Observability Matters GameDay in our offices to ensure our team is equipped with the knowledge and resources they need to quickly and proactively identify and resolve potential issues. We continually evaluate and iterate our efforts to ensure we are providing the best products, solutions and customer support in the video streaming technology industry.” said Somasundaram.

Observing what matters

While talking with customers about their operations, we found a pattern surrounding their AWS observability challenges. Customers either did not know where to start, so they started where they traditionally had in the past – with infrastructure metrics or customers had lots of telemetry and tools in place, yet were still unable to promptly identify root causes, correlate and leverage their observability tools and infrastructure efficiently. In modern applications, you can be more agile, adapt quickly, deploy often, remove monoliths and break down dependencies, but with that came some challenges. There is data everywhere, resources are short lived and it’s hard to correlate across multiple tools, consoles and services.

Each code function, compute instance or database table ultimately serves a business purpose. These will all publish their utilization metrics such as CPU, Memory, Disk and Network, and they are all important. However, we must define metrics for higher level business metrics to understand how those systems and applications are serving their purpose, and delivering on your organization’s vision. So, this is what you should start observing.

With that in mind, to give customers who enjoy GameDay an opportunity to learn about observing what matters in a gamified and fun way, a team of AWS builders came up with this GameDay. The Observability matters GameDay is based on a fictitious Unicorn.Rentals startup, particularly the non-profit unicorn retirement arm. Their vision was to keep the retired unicorns happy and their bellies full, and to do that, they accepted donations of carrots via their website! So, bearing in mind that the happiness of those unicorns is what matters, what do you think we started with observing…. Here is a hint, it’s all about the carrots! Of course, we cover more than JUST business metrics – take a look at the next section which explains a little more about the challenges you can expect in our GameDay.

Observability Matters leads teams through the key pillars of observability: metrics, logs and traces, and through establishing the right tools to gain insights into their systems and application performance. The GameDay consists of 5 Quests with tasks to complete in each Quest. Below is a brief overview of each Quest (without giving too much away of course!).

Observing What Matters

A pre-requisite Quest where players work together to implement solutions and source data which will help them observe their environments and derive the right metrics to observe what matters. Players get to dive deep into log data, metrics and additional features which help them collect the right data to keep them informed if their carrot donations are flowing. It’s not just more metrics, it’s the right kind of metrics to observe the signal for what really matters to you, your business, your project, or your users.

Observe and Optimize

In this Quest, players get to dive deeper into visualizing their telemetry data and using more tools to understand whether the users of their website are having a good experience. Players develop a unified dashboard in order to create actionable insights and context to move from diagnosing the problem to understanding the root cause quicker. Fully understanding business performance often requires external data sources, and players will explore this idea by bringing in an RSS feed to their dashboard.

Amazing Trace

Here, players learn about the importance of tracing when building a distributed system. Such systems can be difficult to debug across many components. Using AWS X-Ray, players learn the value of tracing not only for debugging, but also for gaining insights into their applications. How often have you gotten a call that something is broken but not sure what that is!?

The Edge of Glory

In this Quest, players observe and track their website performance, restrict and gain insights from traffic at the edge, and learn some advanced techniques for processing unstructured log data. It’s important to understand the very beginning of the customer journey of using our system, and the rest of the observability practice yields much less outcome if you don’t monitor the first entry point. The systems are operating well, but the users are not able to access your endpoint!?

Open Sourcery

This Quest showcases how OpenTelemetry provides a mechanism to observe your applications using the tooling that you already know and love. We integrate with open source solutions such Amazon managed service for Prometheus and Amazon Managed Grafana to demonstrate these ideas, and also highlight ways to integrate with managed AWS services.

How to get started

Observability Matters on AWS GameDay is now generally available. Reach out to your AWS account team to get started! Visit the AWS GameDay website for more information about upcoming AWS or Partner-led events and Step Up Your GameDay livestream sessions. Check out the AWS Observability Best Practices guide for prescriptive guidance on AWS observability.

Bio