How do I use AWS WAF to block HTTP requests that don't contain a User-Agent header?

Last updated: 2020-10-22

I want to block HTTP requests that don't have a User-Agent header or have an empty User-Agent header value in the request. How do I use AWS WAF or AWS WAF Classic to block these requests?

Short description

By default, AWS WAF filters don't check if HTTP request parameters are present or not. However, you can create a rule with conditions to check for those parameters.

With AWS WAF, you can use the following conditions:

With AWS WAF Classic, you can use the following conditions:

Resolution

If you use AWS WAF, choose one of the two following options.

Option 1: Create a rule with a regex pattern set

First, create the regex pattern set:

  1. Open the AWS WAF console.
  2. In the navigation pane, under AWS WAF, choose Regex pattern sets.
  3. For Region, select the Region where you created your web access control list (web ACL).
    Note: Select Global if your web ACL is set up for Amazon CloudFront.
  4. Choose Create regex pattern sets.
  5. For Regex pattern set name, enter testpattern.
  6. For Regular expressions, enter .+
  7. Choose Create regex pattern set.

Add the regex pattern set to your web ACL:

  1. In the navigation pane, under AWS WAF, choose Web ACLs.
  2. For Region, select the Region where you created your web ACL.
    Note: Select Global if your web ACL is set up for Amazon CloudFront.
  3. Select your web ACL.
  4. Choose Rules, and then choose Add Rules, Add my own rules and rule groups.
  5. For Name, enter UA-Rule.
  6. For If a request, choose doesn’t match the statement(NOT).
  7. For Statement, choose inspect header.
  8. For Header field name, enter User-Agent.
  9. For Match type, choose Matches pattern from regex pattern set.
  10. For Regex pattern set, choose testpattern.
  11. For Text transformation, keep the default of None.
  12. For Action, keep the default of Block.
  13. Choose Add Rule.
  14. Choose the priority of the rule in the web ACL.
  15. Choose Save.

Option 2: Create a rule with a size constraint condition

Note: By default, the number of regex pattern sets per account is 10. If you've exceeded this AWS WAF limit, you can use the following size constraint solution.

  1. Open the AWS WAF console.
  2. In the navigation pane, under AWS WAF, choose Web ACLs.
  3. For Region, select the Region where you created your web ACL.
    Note: Select Global if your web ACL is set up for Amazon CloudFront.
  4. Select your web ACL.
  5. Choose Rules, and then choose Add Rules, Add my own rules and rule groups.
  6. For Name, enter UA-Rule2.
  7. For If a request, choose doesn’t match the statement(NOT).
  8. For Statement, choose inspect header.
  9. For Header field name, enter User-Agent.
  10. For Match type, choose Size greater than or equal to.
  11. For Size, enter 0 (zero).
  12. For Text transformation, keep the default of None.
  13. For Action, keep the default of Block.
  14. Choose Add Rule.
  15. Select a location in the web ACL to save the rule.
  16. Choose Save.

Note: Use the Core rule set (CRS) in your web ACL. This AWS Managed Rule set contains a rule that blocks requests that don't contain a user-agent. Note that the addition of AWS Managed Rules uses WAF Capacity Units (WCU) of the web ACL.

If you use AWS WAF Classic, choose one of the two following options.

Option 1: Create a rule with a regex matching condition

First, create the regex matching condition:

  1. Open the AWS WAF console.
  2. Choose Switch to AWS WAF Classic.
  3. In the navigation pane, choose String and Regex matching.
  4. Choose Create condition.
  5. For Name, enter UA-condition.
  6. For Region, choose the Region where you created your web ACL.
    Note: Select Global if your web ACL is set up for Amazon CloudFront.
  7. For Type, choose Regex match.
  8. For Part of the request to filter on, choose Header.
  9. For Header, choose User-Agent.
  10. For Transformation, choose None.
  11. For Regex patterns to match to request, keep the default selection of Create regex pattern set.
  12. For New pattern set name, enter testpattern.
  13. Enter a regular expression .+, and then choose the plus (+) symbol.
    Note: The regular expression (regex) matches any character except a line terminator.
  14. Choose Create pattern set and add filter.
  15. Choose Create.

Then, create a rule and add the condition to it:

  1. In the navigation pane, choose Rules.
  2. Choose Create rule.
  3. For Name, enter UA-Rule.
    Note: The Amazon CloudWatch metric name automatically populates based on your entry in the Name field.
  4. For Rule type, choose Regular rule.
  5. For Region, choose the Region where you created your web ACL.
    Note: Select Global if your web ACL is set up for CloudFront.
  6. For Add conditions, choose does not and match at least one of the filters in the string match condition.
  7. Choose UA-condition from the condition drop-down menu.
  8. Choose Create.

Finally, add this rule to your web ACL:

  1. In the navigation pane, choose Web ACLs.
  2. Choose the name of your web ACL.
  3. Select the Rules tab, and then choose Edit web ACL.
  4. For Rules, choose UA-Rule.
  5. Choose Add rule to web ACL.
  6. Confirm that Block is selected for Action.
  7. For Default action, choose Allow all requests that don't match any rules.
  8. Choose Update.

Option 2: Create a rule with a size constraint condition

Note: By default, the number of pattern sets per account is 5. If you've exceeded this AWS WAF limit, you can use the following size constraint solution.

First, create the size constraint condition:

  1. Open the AWS WAF console.
  2. Click Switch to AWS WAF Classic.
  3. In the navigation pane, choose Size constraints.
  4. Choose Create condition.
  5. For Name, enter UA-condition2.
  6. For Region, choose the Region where you created your web ACL.
    Note: Select Global if your web ACL is set up for CloudFront.
  7. For Part of the request to filter on, choose Header.
  8. For Header, enter User-Agent.
  9. For Comparison operator, choose Greater than or equal.
  10. For Size (Bytes), keep the default value of 0.
  11. For Transformation, choose None.
  12. Choose Add filter.
  13. Choose Create.

Then, create a rule and add the condition to it:

  1. In the navigation pane, choose Rules.
  2. Choose Create rule.
  3. For Name, enter UA-Rule2.
    Note: The CloudWatch metric name automatically populates based on your entry in the Name field.
  4. For Rule type, choose Regular rule.
  5. For Region, choose the Region where you created your web ACL.
    Note: Select Global if your web ACL is set up for CloudFront.
  6. For Add conditions, choose does not and match at least one of the filters in the size constraint condition.
  7. Choose UA-condition2 from the condition drop-down menu.
  8. Choose Create.

Finally, add this rule to your web ACL:

  1. In the navigation pane, choose Web ACLs.
  2. Choose the name of your web ACL.
  3. Select the Rules tab, and then choose Edit web ACL.
  4. For Rules, choose UA-Rule.
  5. Choose Add rule to web ACL.
  6. Confirm that Block is selected for Action.
  7. For Default action, choose Allow all requests that don't match any rules.
  8. Choose Update.

Did this article help?


Do you need billing or technical support?