How to Monitor your Applications Effectively

This article is part of a technical content series crafted by AWS Startup Solutions Architects to help guide early stage startups in setting the foundations needed to start building quickly and easily. The series offers a high-level overview of the technical decisions startup founders need to make when getting off the ground, along with which AWS services are best suited to address those decisions.


How to Monitor your AWS Applications

In the process of bringing your startup’s application to the market quickly, “Monitoring,” an area which sometimes gets ignored, is critical for business continuity and to better understand the user experience.

Startups often ask how to trace back an issue in their system or how to monitor their application performance. With application monitoring, you should be able to monitor front-end and back-end performance constantly so that at any given moment, you know the overall performance and health of your application.

In this article, we help you understand the elements of application monitoring and how to achieve better visibility using AWS services. We walk through the three Amazon services developers can leverage to implement application monitoring easily and effectively on AWS.

Why do you need application monitoring?

Today's application environments are highly distributed and in constant flux as they change and expand. Transactions move through multiple servers, and calls may branch out into a handful of threads. When a performance issue arises in any part of this ecosystem of code and network, hardware might be the culprit. Without an application monitoring tool, you have limited performance monitoring options.

Let's take front-end performance for example. The options to gain full visibility into how users are experiencing your application are very limited. You can run test transactions from time to time, or you can just wait for user complaints, which puts you in a reactive mode.

It's also critically important to know the performance of individual business transactions, such as log-ins and check-outs, rather than only monitoring the application as a whole, which tends to hide transaction-specific problems.

Initially, the code we used to debug was in a single machine, and all the logs were going into a single location. Now with distributed applications, however, you have tens, sometimes hundreds of individual components sending logs in potentially varying formats to multiple places. How do you debug this when something goes wrong?

To help you gain this visibility and observability, we recommend using a few AWS services.

Distributed Tracing

AWS X-Ray helps developers analyze and debug distributed applications such as those built using a microservices architecture. With X-Ray, you can understand how your application and its underlying services are performing to identify and troubleshoot the root cause of performance issues and errors. X-Ray provides an end-to-end view of requests as they travel through your application and shows a map of your application’s underlying components.

application-monitoring-1

You can use X-Ray to analyze both applications in development and in production, from simple three-tier applications to complex microservices applications consisting of a large number of services. For example, many startups use Amazon API Gateway as the entry point to their application, whether it’s an AWS Lambda function, an Amazon EC2 instance, or a container in the backend. Enabling X-Ray on API Gateway gives you a full view of the request coming from the user into your application.

But what if you want to monitor how your users experience your application? For example, how much time does it take to load a page? Is a link broken? Are there any page load errors?

Monitor your sites, API endpoints, web workflows

In order to answer these kinds of questions, we suggest using Amazon CloudWatch Synthetics. Amazon CloudWatch Synthetics allows you to monitor application endpoints more easily. You are able to continually verify your customer experience, even when you don’t have any customer traffic on your applications. This enables you to discover issues before your customers do.

CloudWatch Synthetics introduces the concept of canaries. Canaries are configurable scripts that follow the same routes and perform the same actions as a customer. This enables the outside-in view of your customers’ experiences and your service’s availability from their point of view. You can run a canary once or on a regular schedule. Scheduled canaries can run 24 hours a day and as often as once per minute.

How-to-monitor-applications-effectively-2

Customize these tests to check for availability, latency, transactions, broken or dead links, step-by-step task completions, page load errors, load latencies for UI assets, complex wizard flows, or checkout flows in your applications.

How-to-monitor-applications-effectively-3

End-to-end view of your application

To tie all these monitoring tools together in one place, use Amazon CloudWatch ServiceLens. Amazon CloudWatch ServiceLens provides you with a service map that visualizes the contextual linking of all your resources with an intuitive interface so you can dive deep into correlated monitoring data.

How-to-monitor-applications-effectively-4

Amazon CloudWatch ServiceLens enables you to gain visibility into your applications in three main areas:

1. Infrastructure monitoring, which uses metrics and logs to understand the resources supporting your applications.

2. Transaction monitoring, which uses traces to understand dependencies between your resources, and identify faults in your services.

3. End user monitoring, which uses canaries to monitor your endpoints and notify you when your end user experience has degraded.

Conclusion

By using AWS X-Ray to monitor application traces and get a view of your connected services, using Amazon CloudWatch Synthetics to create canaries to monitor your endpoints, and using Amazon CloudWatch ServiceLens to analyze the health of your applications early in your startup journey with AWS, you’ll have no problem identifying and optimizing your application performance as you scale.

Have fun, and build on!