AWS Public Sector Blog

Integrate AI-powered coding assistance in secure environments using Continue and Amazon Bedrock

AWS Branded Background with text "Integrate AI-powered coding assistance in secure environments using Continue and Amazon Bedrock"

Organizations adopting modern software development activities continue to embrace the advantages of AI and large language models (LLMs), maximizing the productivity of developers. Amazon Q Developer provides you with an AI coding companion that delivers direct access for developers to the AI companion within the integrated development environment (IDE).

Passing the context of relevant code from the IDE to an AI tool during software development offers massive productivity gains. However, it also opens the way for data spillage or release of proprietary information unless proper precautions are taken. Organizations that deploy in highly regulated environments, often in Amazon Web Services (AWS) GovCloud (US) Regions, are looking for a way for their developers to securely use approved models outside Amazon Q that have appropriate guardrails and have the potential for customization.

In this post, we walk you through an example you can use leveraging the power of Amazon Bedrock to provide a coding assistant in your IDE.

Architecture overview

At the time of this post’s publication, Amazon Q is not yet available in AWS GovCloud (US) Regions. However organizations can implement a robust alternative by combining the open source AI code assistant Continue with the serverless inference capabilities of Amazon Bedrock, both of which are fully supported in AWS GovCloud (US) Regions. This architecture uses Continue’s extensible framework to interface with Amazon Bedrock to create a secure, compliant AI coding assistant directly in your IDE. Although not discussed or evaluated as part of this example, alternative AI coding tools such as Cline or Aider could be opted for as well.

This example provides several key advantages:

  • Complete data sovereignty within AWS GovCloud (US) Regions
  • Customizable prompt engineering and model selection
  • Integration with existing IDE workflows
  • Ability to implement additional security controls and audit logging
  • Support for air-gapped environments through private API endpoints

By combining these components, organizations can provide their development teams with AI-assisted coding capabilities while maintaining the strict security and compliance requirements of AWS GovCloud (US) Region environments.

Authentication is completed through the standard AWS configuration file on your host machine. This example employs AWS Identity and Access Management (IAM) Roles Anywhere to provide secure, automated credential management. By using X.509 certificates and a private certificate authority (CA), developers can obtain temporary AWS credentials without manual intervention.

In the following configuration, we show Visual Studio Code (VS Code) installed on a local workstation. Alternatively, the IDE can be installed on a virtual environment inside AWS, such as Amazon AppStream 2.0 or Amazon Workspaces to have a preconfigured and scalable environment for developers or system administrators.

Figure 1. High-level example architecture

Prerequisites

To deploy the example, you need access to Amazon Bedrock foundation models (FMs), which isn’t granted by default. You can request or modify access only by using the Amazon Bedrock console. Follow these steps:

  1. Sign in to the target AWS account with an IAM role that has sufficient permissions to manage access to FMs.
  2. Follow the instructions at Add or remove access to Amazon Bedrock foundation models.
  3. For this post, enable access to Anthropic’s Claude 3.0 Haiku and Amazon Titan Text Express.

Deployment walkthrough

To deploy the example, use the instructions in the following sections.

Deploy IAM Roles Anywhere

You can use the Amazon Cloud Development Kit (AWS CDK) code written in Typescript at GitHub to deploy the configurations for AWS Private Certificate Authority, AWS Certificate Manager (ACM), IAM, and IAM Roles Anywhere. The code provided is an initial framework that can be adjusted as necessary. Next, we review some portions of the code.

In this example, AWS Private CA enables creation of Certificate Authority (CA) root and subordinate CAs without the investment and maintenance cost of operating and on-premise CA. The configuration of the root certificate within Private CA can be configured with the specifics for your organization.

    // Create a root Certificate Authority
    const rootCA = new acmpca.CfnCertificateAuthority(this, 'RootCA', {
      type: 'ROOT',
      keyAlgorithm: 'RSA_2048',
      signingAlgorithm: 'SHA256WITHRSA',
      subject: {
        country: 'US',
        organization: 'Your Organization',
        organizationalUnit: 'Your Org Unit',
        state: 'Your State',
        commonName: 'your-domain.com',
        locality: 'Your City'
      },
      csrExtensions: {
        keyUsage: {
          keyCertSign: true,
          crlSign: true
        }
      }
    });

From the root certificate in AWS Private CA, the AWS CDK will to create an example client certificate that can be installed on the client device. The client certificate will be passed through the IAM Roles Anywhere credential helper tool to allow the client to assume the role configured as part of IAM Roles Anywhere.

// Create a client certificate in ACM issued by the Private CA
const clientCert = new acm.PrivateCertificate(this, 'ClientCert', {
  certificateAuthority: acmpca.CertificateAuthority.fromCertificateAuthorityArn(
    this,
    'ImportedCA',
    rootCA.attrArn
  ),
  domainName: 'your-domain.com', // Replace with your domain
  subjectAlternativeNames: ['alt.your-domain.com'], // Optional: add if needed
  keyAlgorithm: acm.KeyAlgorithm.RSA_2048, // Optional: defaults to RSA_2048
});
clientCert.node.addDependency(rootCAActivation);

Below is a guideline to adjust and deploy the example code:

  1. Clone or download the repo locally.
  2. Navigate to the VSCodeBedrock folder.
  3. Adjust any settings in the lib/vs_code_ai-stack.ts file:
    • Region:
      • Set to ‘us-gov-west-1’ as example but can be changed to the desired target region.
    • Root Certificate:
      • templateARN: Change the partition if deploying outside of GovCloud
      • validity value: Adjust for certificate expiration requirements
    • Client Certificate:
      • domainName: Replace ‘your-domain.com’ with your desired domain name
      • subjectAlternativeNames: add any alterntiave names as desired
      • keyAlgorithm: Optional setting, default is RSA_2048
    • Trust Policy for IAM Roles Anywhere
      • String Condition: is set to string value for the aws-us-gov partition, change if deploying in different partition
    • Profile for IAM Roles Anywhere:
      • durationSeconds: how long before the authentication session should time out. Adjust as required.
    • Deploy CDK via ‘cdk deploy’

The created private certificate and associated private key can then be exported to the client devices certificate store that you want to establish the connection to Amazon Bedrock from. In the example we utilize OpenSSL which is an open-source software library for Secure Sockets Layer (SSL) communication. Follow these steps:

  1. On the ACM console, select the private certificate to use on the local workstation.
  2. Set a private encryption passcode and generate Privacy Enhanced Mail (PEM) encodings.
  3. Download the exported certificate body, chain, and private key.
  4. Locally, decrypt the private key with OpenSSL. [OpenSSL Compilation and Installation]

openssl ec --passin pass:<encryptionpassphrase> -in private_key.pem -out private_key_decrypt.pem

We will allow the end user to assume an IAM through IAM Roles Anywhere and limit the permissions of the session to only call the APIs required for this particular use case. The below section of the CDK establishes the role and allows the role to be assumed by IAM Roles Anywhere:

    // Create the role and attach the customer managed policy

    const bedrockRole = new iam.Role(this, 'BedrockRoleAnywhere', {

      roleName: 'bedrock-role-anywhere',

      assumedBy: new iam.ServicePrincipal('rolesanywhere.amazonaws.com'),

      description: 'Role for accessing Bedrock via IAM Roles Anywhere in GovCloud'

    });

Set up Continue

To set up Continue, follow these steps:

  1. Download the Continue plugin for VS Code or JetBrains using the Continue homepage.
  2. After installing the Continue plugin, open the configuration file at ~/.configure/config.json. The configuration file can also be accessed by opening the Continue plugin, selecting the settings icon, and choosing Open configuration file at the top of the screen, as shown in the following screenshot.

    Figure 2. Settings page of Continue for accessing the configuration file

  3. Within the configuration file, modify the “models” object to contain the appropriate Amazon Bedrock models you wish to use. For more information on properly configuring Amazon Bedrock as a model provider for the plugin, refer to Amazon Bedrock under Model providers in the left navigation pane in the Continue documentation.

{
  "models": [
    {
      "title": "Claude 3.0 Haiku",
      "provider": "bedrock",
      "model": "anthropic.claude-3-haiku-20240307-v1:0",
      "region": "us-gov-west-1",
      "profile": "bedrock_dev"
    },
    {
      "title": "Amazon Titan Text Express",
      "provider": "bedrock",
      "model": "amazon.titan-text-express-v1",
      "region": "us-gov-west-1",
      "profile": "bedrock_dev"
    }
  ],

Enable AWS access

To enable AWS access, follow these steps:

  1. Install the AWS IAM Roles Anywhere credential helper from Get temporary security credentials from IAM Roles Anywhere. The credential helper is available for Windows, macOS, and Linux. SHA26 checksums are provided to verify the integrity of the download.
  2. Create a configured profile in ~/.aws/credentials that uses the recently installed signing helper in the following format. Continue will perform authentication to AWS with the credentials located in this file. Because the example uses IAM Roles Anywhere, this credential helper tool will automatically return the rotated credentials each time authentication is necessary, which means you avoid the hassle of modifying the credentials file each time tokens expire.

    [profile developer]
    credential_process = /path/to/aws_signing_helper credential-process \
    --certificate /path/to/certificate \
    --private-key /path/to/private-key \
    --trust-anchor-arn arn:aws-us-gov:rolesanywhere:region:account:trust-anchor/TA_ID \
    --profile-arn arn:aws-us-gov:rolesanywhere:region:account:profile/PROFILE_ID \
    --role-arn arn:aws-us-gov:iam::account:role/role-name-with-path

  3. Verify a successful connection by selecting your intended model and prompting your first question through Continue.

Adapt to your environment

Although the current example provides a solid foundation for secure AI coding assistance, customers may desire additional network security measures to further enhance protection in highly regulated environments such as AWS GovCloud (US) Regions:

  1. AWS PrivateLink can be used to create private, encrypted network connections between your virtual private cloud (VPC) and Amazon Bedrock. Aligning with the Securely Access Services Over AWS PrivateLink whitepaper, this approach does the following:
    • Eliminates direct internet access to AI services
    • Reduces potential attack surfaces
    • Provides an additional layer of network isolation
  2. Amazon API Gateway can be used to employ these advanced security features:
  3. Leverage AWS Security Hub controls associated with implemented services

These modifications can help organizations meet stringent security requirements, including National Institute of Standards and Technology (NIST), Federal Risk and Authorization Management Program (FedRAMP), and Department of Defense (DoD) security standards by providing:

  • Least-privilege access controls
  • Comprehensive logging and monitoring
  • Encrypted service communications

By adopting these proposed network security enhancements, you can develop a more robust and secure approach to AI-assisted coding in highly regulated environments.

Conclusion

This example demonstrates a practical approach to integrating AI-powered coding assistance in highly secure environments such as AWS GovCloud (US) Regions. By combining Continue, Amazon Bedrock, and IAM Roles Anywhere, you can achieve a framework that balances cutting-edge productivity with stringent security requirements.

This architecture provides organizations with a path to safely use AI technologies so developers enhance their productivity while maintaining comprehensive data sovereignty and compliance. As AI continues to transform software development, architectures like these will be critical in bridging innovation and security.

To learn more, visit AWS in the Public Sector. For additional guidance, connect with your AWS solutions architect or contact us today!

TAGS: , , , , , , , , ,
Keith Boaman

Keith Boaman

Keith is a senior solutions architect with AWS supporting U.S. Department of Defense (DoD) customers. He is passionate about helping customers adopt AWS technologies to transform their mission. In his spare time, he enjoys fishing, yard work, and spending time with his wife and two daughters.

Andrew Istfan

Andrew Istfan

Andrew is a solutions architect at AWS and supports public sector customers, primarily in aerospace and defense. He holds eight AWS certifications and has a passion for networking and infrastructure as code. Outside of work, Andrew is often found playing video games, watching captivating space videos, or skiing down mountain slopes.