AWS Partner Network (APN) Blog
Using Amazon Security Lake with New Relic for Threat Detection and Incident Response
By Larbi Belbecir, Sr. Pre-Sales Cloud, Observability & Cyber Solutions Consultant – New Relic
By Wael Kasrawani, Sr. Partner Solutions Architect – AWS
 |
| New Relic |
 |
Threat detection and mitigation for modern software stacks can be a daunting task if not planned correctly. Systems are complex, polyglot, distributed across multiple environments, and constantly evolving. So are vulnerabilities and threats.
As customer environments expand and grow, they typically utilize an increasing number of solutions and tools to fit their many cybersecurity requirements. This can lead to challenges with increased complexity for the cybersecurity teams trying to understand and react to their overall security posture.
This is precisely where Amazon Security Lake with partner integrations can help.
This post highlights the key benefits of using Amazon Security Lake with the associated New Relic integration to improve your security posture with actionable insights. The integration also provides a single pane of glass for performance and security telemetry data across your stack, and supports auto remediation of potential threats.
New Relic is an AWS Specialization Partner and AWS Marketplace Seller that’s a leading cloud-based observability platform built to create more perfect software. Many of the world’s best software and DevOps teams rely on New Relic to move faster, make better decisions, and create best-in-class digital experiences.
Amazon Security Lake Overview
Amazon Security Lake automates the sourcing, aggregation, normalization, and data management of security data across your organization into a security data lake stored in your Amazon Web Services (AWS) account.
A security data lake helps make your organization’s security data broadly accessible, with controls to your preferred security analytics solutions that power use cases such as threat detection, investigation, and incident response.
Key customer benefits include:
- Centralized security data from multiple sources.
- Easily manage data lifecycle policies.
- Derive insights faster across multiple environments.
- Simplify security operations (SecOps).
Amazon Security Lake automatically partitions incoming data from natively-supported AWS services and converts it to a storage- and query-efficient Parquet format. It retains the data in Amazon Simple Storage Service (Amazon S3) in the customer security-owned account, and normalizes it to the Open Cybersecurity Schema Framework (OCSF).
OCSF makes it easier for security professionals to ingest and correlate telemetry data from different sources in an open-source schema. It can be used by many compatible subscribers including AWS-native services such as Amazon OpenSearch Service or subscriber partners such as New Relic.
Technology subscriber partners help you gain insights and address a variety of security use cases—such as threat detection, investigation, and incident response—through turnkey integrations. Service partners can help you build and utilize your security data lake to accelerate time to value.
The following diagram shows the Amazon Security Lake architecture and how subscribers can be integrated with the service to process the required data.
Figure 1 – Amazon Security Lake architecture.
Common Use Cases and Positioning
- Cloud migration: As customers accelerate cloud adoption, they need to consider modernizing existing security processes to cater for on-premises and cloud-native applications; this includes any existing centralized security solutions. An on-premises solution may not be the most efficient way to centralize data as a customer migrates to cloud due to factors such as costs, latency, and speed to react to security. In such a scenario, Amazon Security Lake can be considered integral to the customer migration plan as part of a wider landing zone architecture.
- AWS-native customers: Customers running environments on AWS typically leverage native tools to capture security-related events such as audit logs, access logs, security findings, and threat alerts. Amazon Security Lake consolidates these data points in a unified format within an AWS location. Customers use this source to process security alerts, threat hunting, and anomaly detection, along with adopting a wider security orchestration, automation and response (SOAR) technologies. This can be achieved using AWS-native or technology partner (ISV) tools.
- Audit and compliance: Customers that are subject to compliance regulations typically need to keep records for a set period of time with an ability to query any part of that duration. Amazon Security Lake provides a cost-effective way to do so by leveraging Amazon S3 Glacier for long-term immutable retention while ensuring such logs haven’t been tempered with.
- Modernize existing solutions: Many customers rely on existing platforms to consolidate and store their security data; these vary from basic storage solutions to enterprise third-party solutions. As customers accelerate cloud adoption, they look to improve their agility by modernizing the current solutions and processes. Amazon Security Lake helps consolidate existing tools in a centralized and cost-effective way while providing supported third-party integrations to leverage existing tooling.
New Relic and Amazon Security Lake
As part of their observability strategy, customers can leverage New Relic’s full-stack observability platform and its collection of 100+ turnkey connectors for core AWS services to monitor their workloads through a single pane of glass and proactively detect and address potential issues.
Using the integration for Amazon Security Lake and other AWS security services, customers can proactively correlate relevant data points to detect, triage, prioritize, and mitigate potential threats and improve their overall cloud security posture.
By forwarding relevant OCSF formatted logs to New Relic, customers can leverage centralized log management capabilities to collect, process, explore, automatically correlate, query, discover, and alert on the Amazon Security Lake log data.
As part of that architecture, New Relic periodically forwards all supported log types (currently Amazon Route 53, VPC flow logs, AWS CloudTrail, AWS Security Hub). This subscriber function—triggered by Amazon Simple Queue Service (SQS) events—consumes object notification messages delivered to a dedicated SQS queue and periodically forwarded to New Relic’s secure telemetry log ingestion endpoints.
Figure 2 – Amazon security data lake and New Relic event-based integration.
Most of the required resources are created by Amazon Security Lake (required role, SQS queue, target S3 bucket) and the New Relic Instant Observability quick start, including the New Relic Security Lake Lambda forwarder function that can be deployed from the AWS Serverless App Repository.
In order to proactively detect and address potential threats and assess their compliance, customers can leverage a set of curated dashboards and create alert conditions that can be created with the help of the New Relic Gen AI Observability Assistant.
Figure 3 – New Relic curated Amazon security lake dashboard.
Customers can use an AIOps-based anomaly detection mechanism to analyze the baseline for a given signal and surface any deviations. The payload of such conditions can be customized to add the required context and relayed to the relevant incident response platforms such as ServiceNow or Slack.
Customers can also leverage Amazon EventBridge as a destination to automate the remediation process for given threats.
Conclusion
By implementing Amazon Security Lake, you can benefit from a consolidated view of your security event data, have it stored in a consistent format, use optimized storage costs, and have direct control over data access.
With the New Relic integration, you can rely on a full-stack observability platform and the actionable insights it provides in context to proactively monitor and respond to security incidents. Amazon Security Lake and the associated New Relic integration improve cloud adoption and cybersecurity capabilities, and strengthens the overall cloud security posture.
Refer to the AWS getting started guide for details on how to quickly set up Amazon Security Lake within your environment. You can then configure the New Relic integration by following the steps depicted in the New Relic integration guide.
.
.
New Relic – AWS Partner Spotlight
New Relic is an AWS Specialization Partner and leading cloud-based observability platform built to create more perfect software. Many of the world’s best software and DevOps teams rely on New Relic to move faster, make better decisions, and create best-in-class digital experiences.
Contact New Relic | Partner Overview | AWS Marketplace