Networking & Content Delivery

Enhance your Security Posture and Reduce False Positives using Client JA3 Fingerprint and HTTP Header Order

Information security practitioners use internet protocol (IP) address-based security controls such as block lists and rate-based rules to block malicious traffic. However, blocking malicious traffic solely based on an IP address can unintentionally block legitimate users, resulting in false positives. This is because many users share an IP address behind a network address translation (NAT) or carrier-grade NAT (CGNAT). To address this issue, we can gather supplementary client data through profiling techniques and use it to refine the security controls. Consider this analogy: Rather than identifying a person solely by their name, we can improve identification accuracy by incorporating other attributes like the person’s height and eye color.

In this post, we discuss how you can improve your security posture and reduce false positives by using features of two AWS Services: AWS WAF, which helps you to protect against common web exploits, and Amazon CloudFront, a globally distributed content delivery network (CDN) service that improves performance and security of your applications. Specifically, we discuss how the end user’s JA3 Fingerprint and HTTP Header Order data can be used in AWS WAF rules to enhance your security posture. For detailed guidance on managing false positives in AWS WAF, please refer to Managing false positives in AWS WAF.

Introduction to JA3 Fingerprint and Header Order

JA3 Fingerprint and Header Order provide granular-level data that allows you to identify and profile clients and refine your security controls.

The JA3 Fingerprint is a 32-character hash derived from the TLS Client Hello of an incoming request. This fingerprint serves as a unique identifier for the client’s TLS configuration. Example JA3 Fingerprint: 18ad6e1a77c00d96229ccb62a794860d

The Header Order is derived from the client’s HTTP request. It is the preserved sequence in which the request headers, separated by colons, are sent by the client. Example of Header order: host:sec-ch-ua:sec-ch-ua-mobile:user-agent:sec-ch-ua-platform:accept:sec-fetch-site:sec-fetch-mode:sec-fetch-dest:referer:accept-encoding:accept-language:cookie

Common Use-Cases for using JA3 Fingerprint and Header Order

Let’s walk through three common use cases where utilizing JA3 Fingerprint and Header order can be beneficial. To illustrate the use cases, let’s envision a scenario where you are operating an e-commerce business.

Use case 1: Security and fraud investigations using AWS WAF logs

Your team has observed that your e-commerce application’s login uniform resource identifier (URI) is facing a high volume of unexpected client traffic. To investigate the client traffic, you can utilize the AWS WAF logs.

To access the AWS WAF logs, logging needs to be enabled and sent to a logging destination such as CloudWatch Logs log group. To configure AWS WAF logs to be sent to CloudWatch Logs log group, follow the eight steps outlined in the ‘Setup’ section of Analyzing AWS WAF Logs in Amazon CloudWatch Logs. The CloudWatch Log Insights tab in the AWS WAF console makes it easy for you to analyze your AWS WAF logs. With CloudWatch Log Insights, you can search, analyze, and visualize your logs in real-time to troubleshoot issues and identify trends.

Running the query shown in Figure 1 enables you to analyze client traffic to the login URI. Though the client traffic is sent from multiple IP addresses, they all share the same JA3 Fingerprint. This could indicate an automated account takeover (ATO) attack from a single client device that spreads the requests over many IP addresses to evade detection. Having this granular detail can guide you to implement appropriate countermeasures to mitigate this attack.

AWS CloudWatch Log Insights query that show multiple login requests with the same ja3Fingerprint

Figure 1: Client Traffic to login URI

Utilizing the AWS WAF Fraud Control account takeover prevention (ATP) feature to monitor and automatically mitigate account takeover attacks can be an effective preventive measure against this threat.

  • Query used to find JA3 Fingerprint of client requests to login URI in the above investigation

fields @timestamp, httpRequest.uri, httpRequest.clientIp, httpRequest.country, ja3Fingerprint

| filter httpRequest.uri == “/login”

| parse @message ‘{“name”:”user-agent”,”value”:”*”}’ as userAgent

| limit 25

Employing precise slicing and dicing of data through well-crafted queries enhances the efficiency of your investigation and enables you to uncover critical insights. Below are some sample queries that you can use for conducting investigations using AWS WAF logs.

  • Query to find the top JA3 Fingerprints

fields @timestamp

| stats count(*) as requestCount by ja3Fingerprint

| limit 25

  • Query to find client requests from a specific JA3 Fingerprint

fields @timestamp, httpRequest.uri, action, httpRequest.clientIp, terminatingRuleId, labels.0.name, labels.1.name, labels.2.name, labels.3.name

| filter ja3Fingerprint = “enter-ja3fingerprint-here”

| parse @message ‘{“name”:”user-agent”,”value”:”*”}’ as userAgent

| limit 25

  • Query to find requests that are blocked by a specific WAF Rule

fields @timestamp, httpRequest.uri, action, httpRequest.clientIp, ja3Fingerprint, terminatingRuleId, labels.0.name, labels.1.name, labels.2.name, labels.3.name

| filter action == “BLOCK”

| filter terminatingRuleId == “AWS-AWSManagedRulesSQLiRuleSet”

| parse @message ‘{“name”:”user-agent”,”value”:”*”}’ as userAgent

| limit 25

Use case 2: Implement granular block-lists using AWS WAF

Block-list rules implemented in AWS WAF can be refined using JA3 Fingerprint. You can use the JA3 Fingerprint match statement in AWS WAF to implement the rules. Below is an example of using the JA3 Fingerprint in WAF rule to block known malicious IP addresses.

While analyzing your application’s WAF logs, you notice that one specific IP address is regularly targeting your application with malicious SQL injection and cross-site scripting attacks. This IP address belongs to a leading telecom service provider that employs NAT. Since you also have legitimate users that share the same IP addresses as the malicious clients, you are unable to block this IP address outright.

For this scenario, you can incorporate JA3 Fingerprint in your block-list rule to selectively block the malicious client. You can identify the JA3 Fingerprint using AWS WAF logs and configure a block-list rule as shown in Figure 2 below. This can enable you to accurately block malicious requests while reducing the likelihood of false positives.

AWS WAF custom rule to block traffic using a combination of IP address and JA3 Fingerprint

Figure 2: AWS WAF custom block rule using two parameters

 

Use case 3: Reduce unwanted bot traffic using CloudFront viewer headers and AWS WAF rate-based rules

Unwanted bots are aggressively scraping pricing and content from your application, causing increased cost and performance issues. You have already implemented AWS WAF’s default rate-based rules that block requests when they exceed the configured threshold. However, you get complaints from legitimate customers that they are getting blocked. Further analysis reveals that some of the blocked IP addresses are shared by both unwanted bots and legitimate clients.

By default, AWS WAF’s rate-based rule uses only the client IP address to rate-limit requests. However, you have the option to use other aggregation keys such as HTTP headers, cookies, or query arguments.

To address the problem of legitimate clients getting blocked, you can incorporate JA3 Fingerprint and Header Order data from CloudFront viewer headers to enhance the rate-based rule. JA3 Fingerprint value is provided in the CloudFront-Viewer-JA3-Fingerprint header and Header Order value is provided in the CloudFront-Viewer-Header-Order header. By using the two CloudFront viewer headers as request aggregation keys, you can implement precise rate-based rules to block malicious traffic patterns while minimizing the impact on legitimate users.

 

 Exploring the architecture behind the solution for use case 3

 The following section provides a deep dive of the solution and also provides the steps to implement and test it.

Architecture diagram that shows Amazon CloudFront viewer headers inspected by AWS WAF

Figure 3: Architecture diagram

The workflow of the solution is as follows:

  1. A client accesses your application and requests one or more objects. DNS routes the request to the CloudFront edge location that can best serve the request.
  1. CloudFront checks its cache for the requested content. If the content is in the cache, CloudFront returns it to the user. If not, CloudFront adds the CloudFront-Viewer-JA3-Fingerprint and CloudFront-Viewer-Header-Order headers using the origin request policy and forwards the request to AWS WAF.
  1. AWS WAF inspects the incoming request header according to the configured web access control list (ACL) rules. It can inspect the CloudFront-Viewer-JA3-Fingerprint and CloudFront-Viewer-Header-Order headers, along with other request parameters. It then forwards the request to the Origin. It also sends the logs to Amazon CloudWatch Logs log group.
  1. Amazon CloudWatch can be used to search, analyze, and visualize your logs in real time

 This solution can be implemented using AWS WAF which is associated with supported regional services like Amazon API Gateway and Application Load Balancer. This solution does not work with AWS WAF associated with CloudFront as the WAF inspection here occurs before the CloudFront viewer headers are added.

Steps to implement and test use-case 3 solution

Step 1: Deploy the CloudFormation template

This deploys the services and associated resources required for the solution. It includes the following:

  • An Amazon API Gateway endpoint to be used as the origin.
  • A CloudWatch Logs log group.
  • An AWS WAF web ACL that is associated with the Amazon API Gateway. The WAF has logging enabled to use the configured CloudWatch Logs log group as the destination.
  • A CloudFront distribution that uses the configured Amazon API Gateway as the origin.

a) Launch a CloudFormation stack in your account by clicking the Launch Stack button.

Note: The stack launches in the US East N. Virginia Region (us-east-1). To deploy the solution to another Region, download the solution’s CloudFormation template, provide the required parameters, and deploy it to the selected region.

b) On the Specify stack details page, update the following and deploy the template:

    • Stack name: ja3headerorderstack
    • LogGroupPrefix: ja3headerorderlog

The stack deploys in about 10 minutes. After it finishes, the Output tab lists the following resources as shown in Figure 4.

CloudFormation Output tab providing links to key resources afer the stack is deployed

Figure 4: Output tab of CloudFormation

Step 2: Review the AWS WAF rule

The deployed WebACL of AWS WAF has a rule named “CloudFront_Viewer_Rate_based_rule”. The WebACL can be accessed using the link provided in the Output tab of the CloudFormation stack. The configured rule in the WebACL uses four request aggregation keys, which are CloudFront-Viewer-Header-Order, CloudFront-Viewer-JA3-Fingerprint, URI, and IP address. The rate limit threshold is set to 100. This means that when requests with the same CloudFront-Viewer-Header-Order, CloudFront-Viewer-JA3-Fingerprint, URI, and IP address exceed the threshold of 100 in a five-minute period, they are blocked.

For optimal protection of your applications using this rate-based rule, it’s essential to determine the correct rate-limit threshold to implement. Ensure this threshold is set to avoid blocking legitimate traffic. You can experiment with different rate-limit thresholds with the “Count” rule action to identify the appropriate value. You can also consider using Challenge rule action instead of Block rule action. There is an additional charge for using the Challenge rule action.

Step 3: Test the implementation

To test the configured AWS WAF rate-based rule, you can use a command-line tool that generates a few hundred client requests within a five-minute period.

You can use the CURL command shown below to send 300 requests. You have to replace the CloudFront domain name with your CloudFront domain name. You can find your CloudFront domain name in the Outputs tab of the second CloudFormation stack

for i in {1..300}; do curl -I -X GET “https:// yourcloudfrontdomainname.cloudfront.net/prod/pets/1”; done

In a couple of minutes, after you execute the command, the rate-based rule starts blocking your command-line tool’s test traffic. You can try accessing the same URI from your browser while the WAF is blocking traffic from your command-line tool. You find that your browser’s request is permitted by the WAF. This selective blocking is attributed to the browser having a different JA3 Fingerprint and Header Order value than your command-line tool, which resulted in the browser traffic being allowed.

You can use the below query in ‘CloudWatch Log log insights’ tab in AWS WAF to analyze the JA3 Fingerprint and Header order data from the CloudFront viewer headers.

fields @timestamp, httpRequest.uri, action, httpRequest.clientIp, terminatingRuleId

| parse @message ‘{“name”:”CloudFront-Viewer-JA3-Fingerprint”,”value”:”*”}’ as ja3fingerprint

| parse @message ‘{“name”:”CloudFront-Viewer-Header-Order”,”value”:”*”}’ as headerorder

| limit 25

 

Conclusion

The JA3 Fingerprint and Header Order data provides granular data about the client that enables you to perform deep security and fraud analysis. It also enables you to implement more effective AWS WAF rate-based rules, block-lists, and allow-lists. This can improve your security posture by helping to block your applications from malicious traffic, while allowing access to legitimate clients. Start leveraging JA3 Fingerprint and Header Order data in your AWS WAF configuration today to better protect your applications while maintaining seamless access for legitimate users.

If you have feedback about this post, submit comments in the Comments section below. If you have questions about this post, contact AWS Support.


About the Author

Ram Cholan is an Edge Services Solutions Architect at AWS based in Singapore, where he supports customers to optimize their application performance and improve their security posture. With over 15 years of experience spanning Information Security, Network Engineering, and Managed Services, Ram brings deep technical expertise to enable customers to solve their business challenges.