Open Platform for Enterprise AI - OPEA

1. Run "aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 709825985650.dkr.ecr.us-east-1.amazonaws.com"

2. Run "docker pull 709825985650.dkr.ecr.us-east-1.amazonaws.com/intel/opea-eks-builder"

3. Place a file called "opea.env" in the directory that you are issuing commands from. Add the following required parameters in this format:

AWS_REGION=<your region> AWS_ACCESS_KEY_ID=<your id> AWS_SECRET_ACCESS_KEY=<your secret>" AWS_SESSION_TOKEN=<your token (if assuming a role)> HUGGING_FACE_TOKEN=<your token>

*NOTE: There are sensitive values in this file that should not be shared. Do not copy this file beyond your local drive.

4. (OPTIONAL) Include values in "opea.env" from the Optional Parameters and EKS Cluster sections below as needed

5. Run the command "docker run --env-file opea.env intel/opea-eks-builder:latest"

REQUIRED PARAMETERS

• AWS_REGION - The region you intend to deploy the cluster into

• AWS_ACCESS_KEY_ID - The access key for the user or role you'll be assuming in the AWS account.

• AWS_SECRET_ACCESS_KEY - The secret for the user of role you'll be assuming in the AWS account. If you're assuming a role, you'll also need to set the AWS_SESSION_TOKEN parameter.

*WARNING: Make sure to protect your Access Key and secret values as they are highly sensitive. Be careful not to expose them.

• HUGGING_FACE_TOKEN - A valid token for Hugging Face scoped to use the models in your package. If you're using "guardrails", be sure to add the "meta-llama/Meta-Llama-Guard-2-8B" model to your token scope.

AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, and AWS_SESSION_TOKEN make up your AWS credentials and are required so that you can programmatically authenticate into your AWS account. Once you're authenticated the container image can deploy OPEA ChatQnA into your account on your behalf.

A HUGGING_FACE_TOKEN can be aquired by creating a free account at https://huggingface.co . This will give you access to Hugging Face LLM's needed for OPEA ChatQnA to function.

OPTIONAL PARAMETERS

OPEA_MODULE - If you deploy the package without setting this value, the layout will look like this:

1. Generative AI chatbot user interface
2. Large Language Models (LLM): TGI (Hugging Face; Intel/neural-chat-7b-v3-3)
3. Embedding Models: TEI (Hugging Face; BAAI/bge-base-en-v1.5)
4. Vector Database: (Opensearch)
5. Server (Nginx)

Use the OPEA_MODULE parameter to substitute the defaults with the following replacements:

1. "bedrock" - Amazon Bedrock LLM's. The default is Anthropic Claude 3 Haiku, but you can change the model by setting the LLM_MODEL environment variable.
2. "guardrails" - Monitor the content allowed through the model.
3. "redis" - Use redis as your vector DB

• INSTANCE_TYPE - Supported instance types include any size of the "M7I", "C7I", or "R7I" instances.

• DISK_SIZE - The amount of space (in GB) allotted to nodes in the cluster. We recommend using a minimum of 100.

THE EKS CLUSTER

You can customize the configuration of the EKS cluster that is created, or you can use an existing EKS cluster by passing in the CLUSTER_NAME parameter.

If you choose to bring your own cluster, the settings of the cluster must support the OPEA platform requirements in order to work properly. Also make sure to pass the value for "kubectlRoleArn" so the cluster can run "kubectl" commands.