Documentation · Letting our agent through your CDN · AWS

Allowing our agent through AWS WAF and CloudFront

On CloudFront with AWS WAF, our agent is let through with one rule in your web ACL: an AND statement of an IP set holding our addresses and a user agent that starts with our token, with the action Allow. Its priority puts it after the rules that look for attacks and before Bot Control and your rate based rules.

What to match

• User agent starts with OttomAgent/
• Source address is one of those in https://ottom.io/agent/ips.json

Match both, never the user agent alone: anyone can send our user agent, but only we send it from those addresses. The current addresses are also listed on our agent page.

What you need

• Permission to edit the web ACL attached to your CloudFront distribution.
• The current addresses from https://ottom.io/agent/ips.json.
• Web ACLs for CloudFront live in the Global (CloudFront) scope, so create the IP set there too.

Setting it up

1. 1. Create an IP set

   In the AWS WAF console, open IP sets, choose the Global (CloudFront) scope and create an IPv4 set named ottom-agent. Add each address from the file as a /32.

2. 2. Add a rule to your web ACL

   Open the web ACL attached to your distribution and add a rule of your own with the rule builder. Choose that the request matches all the statements. The JSON below is the same rule, if you prefer the JSON editor.

   {
     "Name": "allow-ottom-agent",
     "Action": {
       "Allow": {}
     },
     "Statement": {
       "AndStatement": {
         "Statements": [
           {
             "IPSetReferenceStatement": {
               "ARN": "<your ottom-agent IP set ARN>"
             }
           },
           {
             "ByteMatchStatement": {
               "FieldToMatch": {
                 "SingleHeader": {
                   "Name": "user-agent"
                 }
               },
               "PositionalConstraint": "STARTS_WITH",
               "SearchString": "OttomAgent/",
               "TextTransformations": [
                 {
                   "Priority": 0,
                   "Type": "NONE"
                 }
               ]
             }
           }
         ]
       }
     },
     "VisibilityConfig": {
       "SampledRequestsEnabled": true,
       "CloudWatchMetricsEnabled": true,
       "MetricName": "allow-ottom-agent"
     }
   }

3. 3. Set its priority

   Allow ends evaluation, so every rule after it is skipped for our requests. Place it below the managed rule groups that look for attacks, such as the core rule set, and above the Bot Control rule group and your rate based rules. In the console, lower in the list is a higher priority number.

4. 4. Save

   Save the web ACL.

Checking it works

Run a scan of your store from your Ottom dashboard. When the rule works, the report has no bot protection finding and the agent reaches your products. In the web ACL’s sampled requests, our requests show allow-ottom-agent as the rule that matched, with the action Allow.

When a scan is still blocked

• You want Bot Control to skip us but every later rule to still run

  Instead of an Allow rule, give the Bot Control rule group a scope down statement: NOT the same AND statement. Bot Control then never inspects our requests, and nothing else changes.

• The rule never matches

  Check the IP set is in the Global (CloudFront) scope, not a region, and that the header name is user-agent in lower case.

Good to know

• The newer AWS WAF console calls web ACLs protection packs. They are the same thing, and the steps are the same.
• An Application Load Balancer or API Gateway in front of your store takes the same rule, with the IP set in that resource’s region instead of Global.

AWS documentation

Menus move. If a name here no longer matches your dashboard, these are the pages we checked this guide against:

• Rule actions
• Processing order of rules
• Using Bot Control, with scope down statements
• Working with the console

Questions

The same rule for other providers, and why it is safe: the CDN guides.