Using Datadog observability features to facilitate modernizing .NET applications on AWS

By Thomas Sobolik – Technical Content Writer at Datadog
      Maxim Brown – Team Lead, Technical Content Editor at Datadog
      Charlie Shen – Sr. Partner Solutions Architect at AWS

Modernization of .NET applications is often a meticulous process, and it benefits greatly by having full visibility into the applications’ behavior. Datadog, a modern, full-stack observability platform, extracts a full suite of data and metrics from your applications. This provides guidance that helps you make informed decisions for modernizing your applications.

Datadog is an AWS Advanced Technology Partner with AWS Competencies in DevOps, Migration, Containers, and Microsoft Workloads. Datadog offers a variety of application and infrastructure monitoring capabilities that help customers quickly search, filter, and analyze metrics, distributed traces, logs, and more. By providing open-ended exploration of data from across your environment, Datadog helps with troubleshooting and optimizing application, platform, and service performance.

In this post, we will explore how you can use Datadog’s automatic .NET instrumentation to gain insights into your application and address key challenges in modernizing a monolithic .NET application to microservices. We will dive deep into using Datadog’s integrations with AWS services and service-level tagging to get visibility across your distributed application.

Modernizing your application

The general idea behind modernizing a monolith application to microservices is to break a single application into multiple, discrete, loosely coupled services to accelerate development and deployment – resulting in faster innovation. While there are many decomposition patterns to choose from, the common core strategy for modernization is to silo and extract logical units from your monolith and redeploy them as isolated microservices.

Breaking monolith to microservices is generally a gradual process, and it’s important to make sure that your application continues to work as expected with each decomposition milestone. By installing the open source .NET tracer and the Datadog Agent on both your monolith and your microservices, you can immediately start seeing request traces and application performance data in Datadog.

Datadog’s .NET tracing library automatically instruments your code along with ADO.NET-compatible databases like SQL Server and PostgreSQL and other data stores like Redis, Elasticsearch, and MongoDB. Datadog has more than 450 built-in integrations, including 75+ AWS service integrations. This means you have visibility into your application requests as they travel across frameworks and other dependencies like distributed caches, data stores, and cloud services in your environment—regardless of how these services are hosted.

To deploy a microservice safely and effectively, you can gradually shift requests from the monolith to the new endpoint. Monitoring traces can reveal whether requests continue to be handled as you expect and where errors or latency are occurring.

A Datadog dashboard that visualizes monitoring data from your monolith alongside your individual services can provide a high-level overview of how your modernization is proceeding. For example, tracking key KPIs like request, error rates, and latency can alert you to any spikes or drop-offs that might indicate that a service is not working properly. Other data visualizations can give you deeper insight into your environment, such as:

• Resource usage metrics, to help determine if your cloud-hosted infrastructure is right-sized for your application
• Logs, to provide additional context around what’s happening in your application and whether there are any spikes in errors
• Synthetic API tests, to ensure that your application’s endpoints are always responsive as you shift traffic over

Figure 1 shows a few of the metrics that are available with Datadog. In this example, application performance and error metrics are simultaneously visible, providing immediate guidance on areas to improve on.

Figure 1. Datadog dashboard displaying application performance and error metrics

Monitoring your microservices

A microservices-based application architecture is inherently more distributed and complex than a monolith. Being able to monitor and visualize data across many discrete services is key to understanding the performance of the whole application and locating issues. However, with each service potentially leveraging its own set of frameworks and technologies, producing its own monitoring data, and deploying from its own CI pipeline, it can be challenging to gather all the information you need to understand a service’s performance.

Let’s look at key Datadog features that help visualize interservice communication and trace requests across multiple dependencies. But first, in order to more effectively connect different types of service telemetry across the Datadog platform, it’s important to use tagging. Tagging helps to connect and collate monitoring data from all your services and their underlying infrastructure.

Unify service data across your environment

Datadog reserves three global tags—env, service, and version—that, when applied to your resources, connect all of their telemetry—including metrics, logs, traces, and more—across Datadog. Applying these tags across your services allows you to easily filter visualizations. It also helps you to navigate all the information you need to understand the health and performance of any subset of your application. You can correlate this data using dashboards, distributed traces, deployment tracking, log analytics, and more.

For example, let’s say we have a microservices-based application that is experiencing high latency. By applying appropriate tags both to our instrumented services and their dependencies, we can seamlessly pivot between request traces and underlying infrastructure metrics to uncover the cause of latency.

Figure 2 shows how you can quickly locate relevant traces by filtering on tags. The global tags are one of the many controllable parameters to help narrow down your search.

Figure 2. Datadog Dashboard showcasing filter controls to locate relevant traces.

Create focused alerts

Alerts are the best way to keep people informed of problems as they arise. Datadog lets you customize the behavior of alerts so that the right people get notified with meaningful issues.

In a microservices architecture, your alerts must effectively identify which services or resources are experiencing the problem. Using tags, individual teams can create alerts that are focused on the services they manage. This means that the relevant team will be automatically notified of a problem reducing alert fatigue for other teams. It helps in shortening troubleshooting time spent identifying who owns the affected systems.

You can use tags to further increase the granularity of your alerts by setting multi alerts, which let you group alerts by tags. In a distributed, cloud-managed environment, this helps to identify the source of problems by alerting you when specific Availability Zones, clusters, or hosts are experiencing an issue.

Figure 3. Customizable error dashboard

Visualize your service interactions

It can be difficult to understand how your microservices interact with each other and isolate where errors are occurring. Datadog’s Service Map automatically maps request traces to visualize the dependency relationships between your microservices in real time.

Visualizing the dependencies between your services provides immediate clarity in the flow of your requests. This makes it simple to identify unexpected communication between services or locate bottlenecks or errors.

Figure 4. Datadog Service Map Dashboard

The Service Map functions as a high-level overview of your application, with visual indicators to identify services that are in an alert state. Highlighting a service node on the map shows the request volume, average latency, and error rate for request traffic between that service and its dependencies. This allows you to quickly identify interservice latency or communication issues. Clicking on a node enables you to immediately pivot to other data scoped to the service in question so you can begin root cause analysis.

Use flame graphs to visualize end-to-end requests

In a microservices environment, individual requests may propagate across dozens of multiple discrete services and dependencies. Having end-to-end visibility into each request as it crosses service boundaries is vital to identifying where slowdowns or errors are happening.

Flame graphs excel at condensing large amounts of data and presenting them in a visual, easy-to-digest manner. Figure 5 shows a flame graph that helps identify latency issues by breaking down the individual stages of a request.

Figure 5. Flame Graph to visualize end-to-end requests

Viewing a trace, you can quickly identify errors (indicated by red boxes around the spans) and latency. As Datadog’s unified service tagging links together your trace data with relevant logs and infrastructure metrics, you can easily correlate infrastructure resource overutilization with latency. You can also dive straight into the logs for an error trace to get more context for debugging.

Monitor service deployments

Microservices architecture facilitates independent development and deployment of each service. This enables teams to deploy much more frequently but makes managing deployments and spotting issues more challenging.

Datadog APM’s Deployment Tracking leverages the version tag to aggregate traces, trace metrics, and logs by code version, enabling you to compare the performance of your deployments in real time.

Figure 6. Datadog deployment tracking dashboard snapshot

Deployment Tracking visualizes key metrics including request volume, error rate, and latency by version, so you can quickly identify if a new deployment has introduced errors or performance regressions. You can easily pivot from deployment tracking metrics to relevant traces and logs that are connected via the service tag. This provides more context about what’s happening within your applications and underlying infrastructure. Then, once you’ve assessed the problem and deployed a fix, you can check back in to validate that the error or regression has been resolved.

Conclusion

Modernizing a monolithic .NET application to a cloud-hosted, microservices-based architecture provides scalability, resilience, and reduces operational overhead. But microservices also come with significant observability challenges, both during the modernization and after.

Using Datadog’s turn-key integrations for dozens of key AWS services, like Amazon RDS, Amazon S3, Amazon EKS, Amazon ElastiCache, AWS Fargate, and more, you can view key infrastructure metrics—such as Fargate CPU and memory usage—alongside higher-level telemetry from your .NET applications. Datadog unifies telemetry from across your environment so that you can monitor every layer of your distributed application using a single pane of glass.

You can start monitoring your .NET apps today by installing the Datadog Agent and the .NET Tracer. Or, if you are new to Datadog, sign up for a trial.

AWS can help you assess how your company can get the most out of cloud. Join the millions of AWS customers that trust us to migrate and modernize their most important applications in the cloud. To learn more on modernizing Windows Server or SQL Server, visit Windows on AWS. Contact us to start your migration journey today.