Empowering Data to Deliver Contextual Analytics with Dynatrace Grail and AWS

By Erick Leon, Sr. Technical Alliance Manager – Dynatrace
By Shashiraj Jeripotula, Sr. Partner Solutions Architect – AWS

Dynatrace

Cloud-native architectures are the backbone of modern digital experiences. Managing these environments and taming the massive amounts of data they produce has grown beyond human ability, however. This, in turn, hinders organizational efficiency and slows innovation.

Traditional cloud monitoring, observability, and security approaches rely on fractured tool sets lacking data context. They also require manual processes for data analytics, which are often too slow and costly for rapidly-changing cloud environments.

To address these challenges, Dynatrace expanded its platform with its new Grail data lakehouse technology. With Grail, the platform provides instant, cost-efficient, artificial intelligence-powered analytics and automation of unified observability, security, and business data at any scale.

This enables organizations to store, process, and analyze the enormous volume and variety of data from modern cloud ecosystems while retaining its context and without structuring or rehydrating it.

In this post, we will explore the powerful combination of Dynatrace’s monitoring and observability platform with Amazon Web Services (AWS) cloud services. The post will cover:

• Introduction to Dynatrace and AWS: An overview of both Dynatrace and AWS, highlighting their significance in modern IT infrastructure.
• Why use Dynatrace with AWS: Explaining the benefits of integrating Dynatrace Grail with AWS, such as enhanced visibility, real-time monitoring, and improved performance optimization.
• Setting up Dynatrace Grail on AWS: Step-by-step instructions on how to set up and configure Dynatrace Grail to monitor AWS resources and applications effectively.
• Key features and capabilities: Deep dive into the features and capabilities offered by Dynatrace Grail in the AWS environment, including automated problem detection, intelligent root cause analysis, and seamless scalability.

By reading this post, readers will gain a comprehensive understanding of how Dynatrace Grail and AWS can work together to optimize performance, increase efficiency, and ensure the reliability of applications and services in the cloud environment. Dynatrace is an AWS Partner and AWS Marketplace Seller that provides software intelligence to simplify cloud complexity and accelerate digital transformation.

What Grail Means for Business and IT

Grail enables organizations to unify and contextually analyze massive datasets with near-instant results, thanks to its familiar query language And massively parallel processing (MPP). Grail leverages a data lakehouse architecture—a data repository that features the flexibility and cost-efficiency of a data lake with the contextual and high-speed querying capabilities of a data warehouse—which organizations can use on Amazon Web Services (AWS) offerings.

Uniquely, Grail in action means there are no more schemas, indexes, and storage tiers to manage. A benefit is teams can conduct exploratory data analysis, for example, without predetermining the kinds of questions they want to ask.

With this unified view, Dynatrace automatically identifies critical risks, pinpoints root cause with causal AI (artificial intelligence that identifies cause and effect between an event and its source, not just correlation), tracks service-level objective (SLO) performance, and helps optimize users’ digital experiences—all in real time and at enterprise scale.

Grail further expands the Dynatrace platform’s capabilities to capture and analyze massive volumes of data, minimize silos, and unify all observability, security, and business data. In turn, organizations can proactively improve digital products, prioritize and resolve problems quickly, explore data their preferred way, and take action such as by orchestrating automation.

Additional use cases include the following:

• Protecting customers and brands by conducting application security forensics to identify, mitigate, and prevent data breaches.
• Improving customer satisfaction and maximizing revenue by querying for ecommerce customers who have not been able to finalize their checkout due to a service outage.
• Enabling more efficient multicloud operations by predicting cloud performance and utilization over time to optimize resource allocation based on user needs.

Dynatrace provides confidence to teams and business leaders that mission-critical applications and systems are resilient.

Value for AWS Customers

It’s important for customers to recognize that modernization is an ongoing practice that isn’t conquered overnight and evolves over time.

As AWS brings emerging technologies in the form of service launches, Dynatrace integrates these services into an intelligent observability platform to help customers get valuable insights into their application environments. This includes the immediate ability to take full advantage of Grail’s capabilities, so joint customers can benefit from its features.

Figure 1 – Grail delivers features that traditional database applications have been missing.

Value for Dynatrace Customers

With the introduction of Grail, Dynatrace opens boundless analytics and automation capabilities without having to disrupt or install additional code packages to existing Dynatrace OneAgent deployments.

Grail delivers greater value by unifying observability, security, and business data from across hybrid and multicloud ecosystems.

Traditionally, data is retained in siloed pools that lack the end-to-end context. Grail powers Dynatrace analytics to uncover the precise answers and deliver intelligent automation that organizations need to drive transformation at scale. Because Dynatrace eliminates the need to rehydrate and reindex data (where data is decompressed and made readily available), teams can operate more efficiently and at the pace business demands.

Figure 2 – Grail delivers precise answers quickly.

Grail Architecture

The Dynatrace data lakehouse introduces a new architectural design to address the challenges of context, cardinality, and interpretation. This architecture offers rich data management and analytics features (taken from the data warehouse model) in addition to low-cost cloud storage systems (which are used by data lakes). All of this is on top of the AWS technology-rich infrastructure.

Figure 3 – Grail marketecture.

Prerequisites

• Dynatrace Intelligent Observability Platform account, or get a free 15-day trial.
• Admin permissions.
• Monitor entities in AWS account that will feed data into Grail.

It’s easy to get started with this setup. From your Dynatrace tenant, navigate to Settings from the left menu. Next, scroll down to Log Monitoring and select Activate Grail for logs and events. Finally, toggle the radio button to Activate logs powered by Grail.

Figure 4 – Ease of activation.

Dynatrace Query Language

Now, in addition to the Grail capabilities mentioned previously, you have the ability to use the Dynatrace Query Language (DQL) to better process logs in a new way.

By navigating to the Logs and Events from the left side menu of the Dynatrace platform, you can query your data with the DQL syntax.

Figure 5 – Sampling demo.

After the DQL is executed, it effectively returns the data that’s needed to answer the question your teams need to troubleshoot, investigate, or validate in real-time all the time.

Figure 6 – Sampling demo.

Troubleshooting

• If you cannot complete the steps to enable Grail in your SaaS tenant, ensure you have the right access with your administrator.
• Ensure you are working on a SaaS Dynatrace tenant and not a self-managed instance, as Grail is not available for managed tenants.
• Ensure you have the proper versions of OneAgent deployed.

About Dynatrace Query Language

In addition to Grail, we have also introduced Dynatrace Query Language (DQL), which offers flexibility in parsing, manipulating, and analyzing data with a familiar syntax. This provides quick results across petabytes of data, accelerates complex ad-hoc analyses, and underpins other new Dynatrace capabilities, including Dynatrace Notebooks and Dynatrace AppEngine.

DQL enables organizations to efficiently find the proverbial needle in the haystack from massive volumes of infrastructure, application, security, and business data. It’s built on top of the Dynatrace platform of automatically discovered entities and their dependencies, which provides organizations with a unified, real-time view of hybrid and multicloud environments, including applications, infrastructure, and user experience.

DQL is flexible and easy to use, enabling organizations to quickly discover the answers they need to make informed decisions at any time. Unlike traditional approaches that can require days for a schema to be updated and data to reindex, DQL enables any query at any time. The language supports a wide range of use cases, including real-time analysis, forensics to uncover historical patterns, and ad-hoc exploratory analytics with visual and collaborative notebooks.

Real-Time Insights

One of the key benefits of DQL is the ability to retrieve real-time insights into an organization’s IT infrastructure. The language supports pipe-based queries and schema-on-read capabilities, enabling organizations to ingest and retrieve high cardinality data in real time, providing up-to-date insights into the performance of IT systems.

By retrieving data in real time, organizations identify issues as they start to occur and take corrective action quickly (including trigger automation) to reduce the impact on users and minimize the cost of degradations and downtime.

Figure 7 – Data-driven dashboards powered by Grail.

Historical Data Analysis

DQL supports historical queries, enabling organizations to immediately query data from the past. Historical data analysis is essential for identifying trends and patterns for performance and security in an organization’s IT environment, enabling users to make informed decisions about future IT initiatives.

By analyzing historical data, organizations can identify areas where improvements can be made, allowing them to further optimize and secure their IT infrastructure, improve efficiency, and reduce costs.

Ad-Hoc Queries

DQL also supports ad-hoc queries, supporting organizations to retrieve and explore data on an as-needed basis. Ad-hoc queries are particularly useful for troubleshooting issues, enabling organizations to quickly retrieve data related to a specific problem and in the unified context of topology and events.

By using ad-hoc queries, organizations can identify the root cause of issues quickly, allowing them to take corrective action before users are impacted.

Figure 8 – Logs processed.

Reduced Manual Efforts

By using DQL, organizations can reduce the time and effort required to retrieve data, facilitating teams to focus on more strategic initiatives.

Conclusion

Dynatrace Grail and Dynatrace Query Language (DQL) are two powerful technologies that come out-of-the-box with the Dynatrace platform.

Dynatrace DQL enables users to extract insights from their observability and security data, making it easy to troubleshoot and prioritize issues, drive targeted action, and create custom dashboards. Grail, on the other hand, expands the Dynatrace platform to ingest even more pertinent data to deliver answers and intelligent automation to drive digital products at scale and in real time.

This enhances Dynatrace’s already leading-edge capabilities that provide real-time anomaly detection and automatic root-cause analysis using Davis AI, further reducing the time it takes to troubleshoot issues and improve the accuracy of detection. By combining these technologies, users can streamline their operations and gain more efficiency under one platform with Dynatrace.

To learn more about how Dynatrace and AWS are better together, join us for an upcoming Immersion Day workshop. You can also learn more about Dynatrace in AWS Marketplace.

.

.

Dynatrace – AWS Partner Spotlight

Dynatrace is an AWS Partner that provides software intelligence to simplify cloud complexity and accelerate digital transformation.

Contact Dynatrace | Partner Overview | AWS Marketplace