AWS Open Source Blog
Deploying Open Policy Agent (OPA) as a sidecar on Amazon Elastic Container Service (Amazon ECS)
Introduction
The sidecar deployment pattern lets developers decouple monolithic applications into separate processes with high levels of isolation and encapsulation. To address cross-cutting concerns like logging, monitoring, and authorization, organizations can decouple these operations into sidecar containers shared across multiple microservices within a deployment. In order to perform operations like authorization, microservice deployments often depend on external services or lengthy database queries to make authorization decisions. By using a policy engine like Open Policy Agent (OPA) for authorization, microservices deployments eliminate latency-intensive network calls and deliver authorization decisions within milliseconds.
This blog post will demonstrate how Open Policy Agent (OPA) can be deployed in a sidecar pattern to provide authorization decisions for microservices deployed on Amazon Elastic Container Service (Amazon ECS).
Deploying Open Policy Agent as a sidecar using Amazon ECS
Amazon ECS is a fully managed container orchestration service that helps you deploy, manage, and scale containerized applications. Amazon ECS supports Docker container orchestration using a managed fleet of Amazon Elastic Compute Cloud (Amazon EC2) instances, or using a fully managed, serverless AWS Fargate cluster.
Open Policy Agent (OPA) is an open source, general-purpose policy engine that lets you specify policy as code and provides simple APIs to offload policy decision-making from your applications.
Let’s build a sample web application that receives the caller’s identity in a combination of headers and converts them as structured context data for OPA to make a policy decision that will authorize or deny access to your data.

The diagram above demonstrates the authorization workflow that will be implemented in this blog:
- Users first make an HTTP GET request to an Application Load Balancer (ALB) endpoint with attributes stored in the header defining their assigned group and the resource identifier to be accessed.
- The Application Load Balancer will route requests into an Amazon Elastic Container Service (Amazon ECS) Cluster containing the Primary and Sidecar containers deployed in a sidecar pattern.
- Incoming requests will first reach the primary container, where a NodeJS web service uses the group and resource values stored in the request header to form a policy request.
- The web service will then send the policy request to the sidecar container running Open Policy Agent (OPA) to evaluate the request.
- The OPA daemon receives the incoming request and performs a policy evaluation against stored data and policy documents.
- Once the policy evaluation is complete, OPA will return a boolean (true/false) policy response to the web service.
- Depending on the policy response, the web service will return an authorization decision to the requesting user with an HTTP response stating that the user is either Authorized or Not Authorized to access the requested resource.
Project setup and walkthrough
Prerequisites
This walkthrough assumes the following two steps have already been completed.
- Install AWS Command Line Interface (AWS CLI) version 2 on your local machine.
- Install Docker on your local machine.
Primary container (web service) build steps
First, create a folder called helloopa-service
where we will store the files and folders for the web application that we’ll building in the following steps.
Within the helloopa-service
folder, create a file called app.js
and copy the code below into this file. This web application exposes an API path called /request
that requires two header values: “group” and “resource”. The webservice forwards this request to Open Policy Agent which will evaluate and returns the authorization decision back to the web service to perform any additional operations. Additionally, the web service exposes a /health
path returning a HTTP 200 response for Amazon ECS health checks to complete successfully.
Next, create a file called package.json
in the same folder as app.js
, and paste the following snippet into the file. This will define structure of the web application and will ensure the dependencies required by the web service are installed.
Finally, you’ll create a file called Dockerfile
within the same folder, pasting the following snippet into the file. This will be used to create a container for the web application with all of the necessary steps and requirements for a successful deployment.
# Build NodeJS Web Service directory, Install Dependencies, Run Service
FROM node:16-alpine
RUN mkdir -p /app/node_modules
WORKDIR /app
COPY . .
RUN npm install
EXPOSE 80
CMD ["node", "app.js"]
Sidecar container (OPA) build steps
Create a new folder called opa-service
. Within it, create a folder called data
where you will store the data and policies required by OPA.
First, we’ll create a JSON data file called data.json
in the data directory, which will provide contextual information to OPA for policy evaluation. The following data describes the permissions policy for resources.
file1
can be accessed by users that are members of eitherGuest
,Dev
, orAdmin
group.file2
can only be accessed by users that are members ofDev
orAdmin
groups.file_secret
can only be accessed by users that are members of theAdmin
group.
Next, we’ll create an OPA policy file inside the data
folder called policies.rego
, written in the Rego language, and paste the following code snippet into the file. This policy will deny access to all requests by default, but will allow access only if the user’s group matches a group with valid permissions to access the requested resource.
package opablog
default allow=false
allow=true{
input.group == data.GroupPermissions[input.resource][_]
}
Finally, you’ll create a file called Dockerfile
within the opa-service
folder, and paste the following code snippet into the file. This file will be used to create a container for the OPA service with all of the necessary data and policies and uses the latest release of the OPA executable.
# Build OPA Service directory, load policies and data, install and run OPA daemon
FROM alpine:latest
RUN apk --no-cache add curl
ADD $PWD/data /data
VOLUME /data RUN curl -L -o opa https://openpolicyagent.org/downloads/latest/opa_linux_amd64_static
RUN chmod 755 ./opa EXPOSE 8181
CMD ./opa run -s ./data --skip-version-check
Build and push containers to Amazon Elastic Container Registry (Amazon ECR)
For these next steps, make sure that you’ve installed the AWS CLI and Docker onto your local machine. Also make sure that you’ve configured your AWS CLI with your AWS Account Credentials.
First, create a file outside of the helloopa-service and opa-service folders called called build-push-ecr.sh
, and copy the following code snippet into the file. To execute the script use chmod 755 build-push-ecr.sh
in your terminal to set the correct permissions to the file. This script creates the Amazon Elastic Container Registry (Amazon ECR) repository, builds the specified Docker image locally, and then pushes that image to Amazon ECR.
Note: If you are using an M1-based Mac, you have to use Docker Buildx instead of docker build
. If you are building for x86-based systems use --platform linux/amd64
, and for ARM-based systems use --platform linux/arm64
. In the script above, make sure to comment out docker build, docker tag, and docker push commands and add the following command instead.
docker buildx build -t ${RepoURI}:latest --platform linux/amd64 --push .
Primary container (web service)
- First, we’ll be building and pushing the helloopa-service container. To do this, run the script we’ve created in your terminal by entering
./build-push-ecr.sh [AccountID] [AWSRegion] helloopa-service
.
Sidecar container (OPA service)
- Next, we’ll be building and pushing the opa-service container. To do this, run the script we’ve created in your terminal by entering
./build-push-ecr.sh [AccountID] [AWSRegion] opa-service
.
AWS CloudFormation templates
To automate the deployment of all the necessary resources for this project, we’ll be using three AWS CloudFormation templates. Create a new folder called templates
outside of your helloopa-service
and opa-service
directories. This new folder is where you will create and store the following AWS CloudFormation templates.
helloopa-root stack
Inside the templates
folder, create a file called helloopa-root.yml
and copy the following code snippet into your file. This template defines all the necessary parameters and will call the other two nested stacks in sequence to complete this deployment.
helloopa-network stack
Next, we’ll create a file called helloopa-network.yml
and copy the following code snippet into the file. This stack will deploy all of the required networking, security, and cluster resources into your AWS environment.
helloopa-service Stack
For our third stack, create a file called helloopa-service.yml
and copy the following code snippet into the file. This stack will configure load balancing to our ECS cluster, define the service and task definition for our deployment, and will configure each of the containers themselves.
Deployment
Now that you’ve created all three CloudFormation templates, create an Amazon Simple Storage Service (S3) bucket for your CloudFormation templates on your AWS account. To deploy the stacks that you just created, create a file called “package-deploy-cfn.sh” and copy the code snippet below into the file. Make sure to set execute permissions on the file by running chmod 755
package-deploy-cfn.sh
. Make sure to modify the script with the parameters you want to use for VpcCIDR
, PublicCIDRA
, and PublicCIDRB
under --parameter-overrides
.
Run the following script ./package-deploy-cfn.sh
with your AWS Region, S3 bucket name, root stack name, and the name of the environment you want to deploy resources into.
Testing
Once you AWS CloudFormation shows CREATE_COMPLETE
for the helloopa-root
stack, all of your resources have been successfully deployed into your AWS environment.
To start testing your deployment, first navigate to the “Outputs” section of the helloopa-root
stack and save the ApiUrl
value. In the following CURL commands, replace the ELBADDRESS
with the ApiUrl
value you just saved. Using the command line on your local machine, run each command and verify that the result you receive matches the expected result value.
To reference why each of these commands evaluate the way that they do, refer back to the “Sidecar container (OPA) build steps” section above.
Test case 1 — Guest accessing file1:
# Expected Result: True (don't copy this line into terminal) curl --location --request GET 'ELBADDRESS/request' \ --header 'group: Guest' \ --header 'resource: file1'
Test case 2 — Guest accessing file2:
# Expected Result: False (don't copy this line into terminal) curl --location --request GET 'ELBADDRESS/request' \ --header 'group: Guest' \ --header 'resource: file2'
Test case 3 — Dev accessing file2:
# Expected Result: True (don't copy this line into terminal) curl --location --request GET 'ELBADDRESS/request' \ --header 'group: Dev' \ --header 'resource: file2'
Test case 4 — Dev accessing file_secret:
# Expected Result: False (don't copy this line into terminal) curl --location --request GET 'ELBADDRESS/request' \ --header 'group: Dev' \ --header 'resource: file_secret'
Test case 5 — Admin accessing file_secret:
# Expected Result: True (don't copy this line into terminal) curl --location --request GET 'ELBADDRESS/request' \ --header 'group: Admin' \ --header 'resource: file_secret'
Troubleshooting
- If you receive an “undefined” response after running any these commands.
- In your AWS Console, navigate to Amazon ECS using the search bar, and make sure both of your containers are running correctly under the service definition.
- You can also navigate to Amazon CloudWatch and check the logs for your Amazon ECS environment that you have configured, checking to see if the OPA container is receiving and responding to requests.
Cleanup
To clean up your deployment, navigate to the AWS Console and search for AWS CloudFormation. From there, select the helloopa-root
stack and select Delete. Next, navigate to Amazon ECR, and delete the ECR repositories you created. Finally, make sure to delete the Amazon S3 bucket containing your AWS Cloudformation templates that you created in Deployment section.
Conclusion
In this post, we built a sample web application that offloads authorization decisions to a sidecar container running Open Policy Agent (OPA) as a sidecar on Amazon ECS. The OPA evaluation speed is possible because OPA keeps contextual data and policies in memory. When deployed as a sidecar, calls made to OPA are local host calls, bypassing any possible network latency.
For use cases where policies and context data need to be updated or to be dynamic, there are different methods to integrate with external data.
The AWS CloudFormation templates in this post can be extended to deploy multiple microservices, each using OPA sidecar containers with policies and context data specifically required for that service.