AWS Partner Network (APN) Blog
Container Threat Detection and Response for AWS Fargate with Sysdig
By Victor Hernando, Sr. Technical Marketing Manager – Sysdig
By Ranjit Kalidasan, Sr. Solutions Architect – AWS
 |
| Sysdig |
 |
Organizations are moving to containerized environments at exponential speeds to free developers from day-to-day host-management so they can focus on building and deploying modern applications.
AWS Fargate on Amazon Elastic Container Service (Amazon ECS) provides the underlying infrastructure needed to run workloads and services, making it attractive for serverless computing. However, the nature of serverless computing requires a shift in instrumentation to gain visibility into containers to monitor and respond to threats.
In this post, we explore key considerations for achieving real-time visibility into AWS Fargate workloads to help identify and minimize risks.
Sysdig is an AWS Specialization Partner and AWS Marketplace Seller with AWS Competencies in DevOps, Containers, Cloud Operations Software, and Security. It provides real-time, cloud-native security at scale to help customers migrate and modernize their container and cloud workloads on AWS.
Why Serverless Containers Need Additional Security
AWS Fargate reduces security risks for workloads and services. The underlying infrastructure is maintained by AWS and improves security through workload isolation by design.
However, in the AWS Shared Responsibility Model containers and their processes running on AWS Fargate tasks are the responsibility of developer teams—not AWS alone.
Since serverless services are accessible from the internet, a precise, rapid threat detection and response for AWS Fargate is required to further minimize risk of outside threats.
When hosts and other basic assets like networking and storage are abstracted away in a serverless model, there’s no way to deploy security instrumentation at the host infrastructure level.
Because AWS manages the infrastructure that powers AWS Fargate, users don’t have access to nodes and, therefore, neither does a third-party security application or agent. This makes it a challenge to monitor and protect workloads with traditional agents.
Threat detection and response for serverless applications on AWS Fargate requires an alternative way to observe serverless workload behavior in real-time. Ideally, a solution will not require you to alter or add code into your container images, yet will be able to deliver deep insights about container activity.
Threat Detection for AWS Fargate on Amazon ECS
At Sysdig, the preferred way to get real-time container and host insights is by pulling insights straight from the kernel. In Linux, an extended Berkeley Packet Filter (eBPF) runs programs in a privileged context, enabling the collection of low-level kernel information by running code within a virtual machine (VM) in the kernel.
Typically, an agent instruments every host and uses an eBPF probe to capture system calls. With AWS Fargate, there’s no visible host to the user. In this case, we need an alternative method for serverless instrumentation to achieve the same level of runtime security visibility.
With AWS Fargate, AWS provides a way to achieve kernel system call tracing by adding the SYS_PTRACE capability. This enables security tools to perform system call tracing which is used for comprehensive workload threat detection.
Falco Threat Detection and AWS Fargate
Falco is an open source, cloud-native security tool created by Sysdig that detects security threats in real-time for Linux and containerized applications. It typically uses either eBPF, open-source kernel modules built on libscap and libsinsp C++ libraries, or Linux user space instrumentation.
To provide visibility into abnormal behavior, potential security threats, and compliance violations for AWS Fargate, Falco leverages user space instrumentation that taps into SYS_PTRACE. This instrumentation collects kernel insights at runtime via system calls, parses the data, and then matches detected security events with declared event definitions within its rules engine.
With this approach, AWS Fargate users can deploy Falco to detect workload security threats in real-time.
Figure 1 – AWS Fargate syscall instrumentation with Sysdig and ptrace.
Falco rules are assets containing the security events to be checked by Falco. A Falco rule file is a YAML file containing the three main elements: rules, macros, and lists.
Sysdig Secure for AWS Fargate
Sysdig Secure is a cloud-native application protection platform (CNAPP) that provides threat detection built on Falco. With it, AWS users can detect unexpected activity for AWS Fargate containers.
In addition, Sysdig adds additional capabilities beyond what Falco alone provides, including:
- SaaS-based security: Users can manage AWS Fargate security at scale along with other types of cloud workloads including Kubernetes and cloud services.
- Out-of-the box policies: Included policies cover a wide variety of cloud and container runtime security use cases for AWS Fargate, Amazon ECS, Amazon Elastic Kubernetes Service (Amazon EKS), and cloud threat detection for AWS services using insights from AWS CloudTrail.
- Automatic remediation: Security policies provide immediate response to detected container threats.
- Forensics investigation: Capture files and record insights from before, during, and after container events to help practitioners understand security incidents.
- Visualizations and dashboards: Threat detection, investigation, and risk prioritization visualizations provide a clear picture into security issues to accelerate response.
Figure 2 – AWS Fargate threat detection in Sysdig Secure.
Sysdig incorporates “shift left” and “shield right” security concepts to enable security from the beginning of the application development process through runtime:
- Shift left is an approach that moves security testing into the early phases of the application development to identify and address vulnerabilities and security flaws.
- Shift right security emphasizes mechanisms to protect and monitor your running services to address threats that arise in production.
By incorporating both of these practices, AWS users can more effectively manage security across the lifecycle of containers from development through runtime.
Sysdig Secure Instrumentation for AWS Fargate
Sysdig uses insights from system calls to detect threats at runtime. Like Falco, Sysdig Secure uses the SYS_PTRACE to gain AWS Fargate visibility.
To support AWS Fargate, Sysdig developed advanced Fargate instrumentation to provide the level of detail and data needed for real-time protection. The instrumentation has helped AWS Fargate customers enable runtime protection for workloads at scale.
The Sysdig advanced AWS Fargate instrumentation is made up of two main components: serverless workload agent, and serverless orchestrator agent.
- Serverless workload agent: Each AWS Fargate workload has a lightweight, serverless workload agent container running in the same AWS Fargate task. This agent is responsible for monitoring container activity. It sends its collected information to the serverless orchestrator agent, which accepts network connections on TCP port 6667.
- Serverless orchestrator agent: The serverless orchestrator agent is a standalone container that runs as a single task in the AWS Fargate environment. It collects security event information from the serverless workload agents and pushes policy updates to the workload agents. The orchestrator sends the information collected from the workload agents to the Sysdig SaaS backend using the network TCP port 6443.
With regards to AWS networking, one orchestrator is required per virtual private cloud (VPC). In addition, the serverless workload agents pushing data to the orchestrator must be in the same VPC as the Sysdig orchestrator.
The Sysdig Agent for AWS Fargate can be easily deployed with either a CloudFormation template or Terraform.
The values for the Sysdig orchestrator stack include:
- AWS network assets: VPC and subnets for the underlying infrastructure.
- Sysdig configurations: Access key to access the Sysdig collector endpoint or the collector host address, which may be required depending on the region where the Sysdig service is hosted.
Figure 3 – Required parameters for the Sysdig orchestrator stack.
The Sysdig workload instrumentation stack requires the following values:
- Orchestrator Agent Host: Provided at Sysdig Orchestrator stack deployment time.
- Orchestrator Agent Port: The default port is 6667; for custom deployments, any new port must be provided in this field.
- Macro Name: The name of the macro must be unique in the AWS account.
- Sysdig Serverless Patcher Image: It must be hosted in an Amazon Elastic Container Registry (Amazon ECR) private repository within the same AWS region in which the deployment takes place.
Figure 4 – Workload instrumentation CFT stack.
When complete, note the “SysdigTransformationMacro” string in the “Output” tab; the value should appear as: Transform: [“SysdigMacro”].
Prior to deploying the user workload, it has to be instrumented by appending this new transformation macro to the root level of its CloudFormation template at deployment time.
At workload deployment time, the transformation macro will do its part by invoking the Sysdig instrumentation stack, which will own the transformation tasks on the workload AWS Fargate task definitions.
This instrumentation happens automatically without requiring further actions from AWS users.
Figure 5 – Workload instrumentation example.
Use Case: Detecting and Responding to Security Events
Security teams need insights into activity to both manage threats as well as to help ensure their organization is following best practices. Let’s look at an example of using Sysdig Secure to detect events using the Log4Shell. This scenario is described in detail in this Sysdig blog post.
The Log4J vulnerability, CVE-2021-44228, allowed the retrieval of objects from a remote or local machine that could then execute arbitrary code. The chain of events begins when an unauthorized user accesses the system by establishing the reverse shell. When this occurs, Sysdig Secure raises a new security event.
Figure 6 – Sysdig Secure detected suspicious network tool event.
In another recent issue discovered by the Sysdig Threat Research Team, a sophisticated cloud operation dubbed SCARLETEEL exploited a container workload to perform privilege escalation.
It made possible a number of unauthorized activities, such as running a crypto miner, finding credentials, and accessing proprietary data. Sysdig’s threat detection for AWS Fargate alerted the team so they could launch a response.
Activity details surrounding an event are also provided, enabling practitioners to dig deeper and explore what happened using the Sysdig Inspect viewer.
Conclusion
In this post, we described the necessity of risk assessments and quick response times in containerized environments. This identification and response process is key to reducing the impacts of unexpected incidents.
Tools such as Falco and Sysdig Secure can help security teams identify and understand risks and take action on AWS Fargate security events to protect their cloud workloads. You can learn more about Sysdig in AWS Marketplace.
.
.
Sysidg – AWS Partner Spotlight
Sysdig is an AWS Specialization Partner that provides real-time, cloud-native security at scale to help customers migrate and modernize their container and cloud workloads on AWS.