Simplifying Kubernetes Observability with Amazon EKS Blueprints
By Brad Schmitt, Principal Solutions Architect – New Relic
By John Withers, Principal Product Marketing Manager – New Relic
By Anuj Sharma, Sr. Specialist Solutions Architect – AWS
Understanding the performance of your Amazon Elastic Kubernetes Service (Amazon EKS) clusters and applications is critical, but without the proper tools identifying problems can be difficult. You need the ability to detect, analyze, and resolve them as fast as possible.
With Kubernetes, a platform-wide observability tool that ties the performance of your services to the underlying cluster infrastructure is vital to your success. By correlating your services’ golden signals with Kubernetes infrastructure, and with container logs that are one click away, New Relic drives down the time it takes you to detect and diagnose issues.
This add-on simplifies deploying Kubernetes observability components from New Relic to your EKS clusters, using programming concepts you’re already familiar with. Additionally, with Pixie built-in to the add-on as an optional component, you get observability using eBPF (Extended Berkeley Packet Filter) with no additional setup required.
The release benefits both infrastructure engineers and developers.
Benefits for infrastructure engineers:
- Turnkey deployment that simplifies the standard Helm install.
- Frictionless upgrades across all clusters as part of a deployment pipeline or via CDK deploy.
- Analyze and alert on node capacity and utilization metrics.
Benefits for developers:
- Analyze and alert on status and utilization metrics for Kubernetes workloads running in the cluster.
- Collect Kubernetes events to identify resource constraints, scheduling issues, etc.
- One-click access to container logs for easier troubleshooting.
- Combine with New Relic APM for correlation between application, infrastructure, and log data.
Choose Your New Relic EKS Observability Components
New Relic’s EKS Blueprint add-on gives you control over what you want to monitor in the cluster.
The configuration outlined in this post will deploy the following components to your cluster. To consider additional options, explore the New Relic EKS Blueprints Addon GitHub repo for more details and examples.
|Component name||What it provides|
|Infrastructure||Node utilization and Kubernetes metrics (kubelet, cAdvisor, and kube-state-metrics)|
|Kube events||Cluster-wide Kubernetes events|
|Logging||Preconfigured instance of Fluent-Bit|
|Prometheus OpenMetrics integration||Metrics from scraped Prometheus endpoints|
|Pixie||Continuous application profiling and protocol-level tracing for HTTP, gRPC, DNS, Kafka, MySQL, PostgreSQL, Redis, and more|
Watch this video to learn how to deploy New Relic’s EKS Blueprints add-on in your cluster. For more detailed instructions, follow along with the steps below the video.
Complete these steps to deploy the New Relic EKS Blueprints add-on:
- If you’re not already a New Relic customer, sign up for free.
- Get your New Relic license key. If you plan to use AWS Secrets Manager, create a secret called
newrelic-pixie-combinedand store the key in JSON format:
Figure 1 – Create license keys in JSON format.
Note that a New Relic license key and cluster name are required parameters for the integrations to work properly. The add-on supports the use of AWS Secrets Manager for storing and retrieving the license key.
- Follow the steps in the Amazon EKS Blueprints Quick Start documentation which may be updated periodically:
- Install the CDK:
npm install email@example.com
- Verify the installation:
mkdir my-blueprint && cd my-blueprint
- Create a new typescript CDK project in an empty directory:
cdk init app --language typescript
- Install the following node package manager packages
npm i @aws-quickstart/eks-blueprints
npm i @newrelic/newrelic-eks-blueprints-addon
- Replace the contents of your
bin/<your-main-file>.tsfile with the following code:
- Install the CDK:
What the Code Does
bin/<your-main-file>.ts, the code accomplishes these tasks:
- Imports the required npm packages, including
- Instantiates a cluster blueprint object.
- Adds the Metrics Server, and Cluster Autoscaler EKS Blueprints add-ons. Additional add-ons can be deployed to the cluster, as outlined in the add-ons docs page.
- Adds the New Relic EKS Blueprints add-on and enables the recommended New Relic components, such as infrastructure monitoring, logs, Prometheus OpenMetrics integration, Kubernetes events, and more.
- Optionally, it can also install Pixie into the cluster through the New Relic add-on.
Deploy Your Cluster
- From the command line, run
cdk bootstrapto provision resources for AWS CDK. For more details around this process, see the AWS CDK Bootstrapping docs.
- Run a
cdk deploycommand to build the cluster using the add-on defined in the example code. It may take 15 minutes or more for the cluster to build in Amazon EKS.
- After the cluster is built, you’ll see some outputs in your terminal that contain an
aws eks update-kubeconfigcommand to update your kubectl context. Update your context, and then validate everything is running successfully in your cluster.
Figure 2 – Validate pods are running.
- Go to your cluster list in New Relic:
- If necessary, select your account under All Accounts.
- Select your cluster name to navigate to the Kubernetes cluster explorer. You’ll see the nodes and pods of your Kubernetes cluster, as show in this screen capture:
Figure 3 – View your nodes and pods in Kubernetes cluster explorer.
To learn more, see Navigate the Kubernetes cluster explorer.
- (Optional) If you deployed Pixie, select the Live Debugging with Pixie tab to access the Pixie integration.
Figure 4 – Live debugging with Pixie.
Observability Made Easy
That’s all it takes to install the AWS CDK, initialize a new project, and build an EKS Blueprint cluster containing the New Relic EKS Blueprints add-on for Kubernetes observability.
Whether you’re just getting started with EKS Shared Services Platforms and AWS CDK, or you’re a mature DevOps shop looking to manage, secure, and automate your development platforms, the New Relic EKS Blueprints add-on gives you deep visibility into the performance of your cluster.
Or sign up for a free New Relic account. Your account includes 100 GB/month of free data ingest, one free full-access user, and unlimited free basic users.
New Relic – AWS Partner Spotlight
New Relic is an AWS Containers Competency Partner and leading cloud-based observability platform built to create more perfect software.
*Already worked with New Relic? Rate the Partner
*To review an AWS Partner, you must be a customer that has worked with them directly on a project.