AWS Compute Blog
Migrating AWS Lambda functions from the Go1.x runtime to the custom runtime on Amazon Linux 2
Update Feb 13, 2024: Support for the provided runtime family, as well as 3rd-party container images, was added in the v1.18 release of aws-lambda-go. Functions that have not had their dependencies updated since July 2020 will need to be re-compiled to prevent invocation errors when using the provided.al2
or provided.al2023
runtimes. If you prefer not to rename your executable to bootstrap
, you can instead create a shell script called bootstrap
and use that to run your executable. Be sure to set the correct permissions on your bootstrap file.
Update Dec 15, 2023: Lambda has since added Amazon Linux 2023 to the provided
runtime family. We recommend that customers upgrading from the go1.x
runtime use the new provided.al2023
runtime as their first preference. The provided.al2
runtime remains available and will be supported until at least June 30, 2025.
This post is written by Micah Walter, Senior Solutions Architect, Yanko Bolanos, Senior Solutions Architect, and Ramesh Mathikumar, Senior DevOps Consultant.
This blog post describes our plans to improve performance and streamline the user experience for customers writing AWS Lambda functions using Go.
Today, customers using Go with Lambda can either use the go1.x
runtime, or use the provided.al2
runtime. Going forward, we plan to deprecate the go1.x
runtime in line with the end-of-life of Amazon Linux 1, currently scheduled for December 31, 2023.
Customers using the go1.x
runtime should migrate their functions to the provided.al2
runtime to continue to benefit from the latest runtime updates and security patches. Customers who deploy Go functions using container images who are currently using the go1.x
base container image should similarly migrate to the provided.al2
base image.
Lambda remains committed to supporting the Go programming language. Using the provided.al2
runtime offers several benefits over the go1.x
runtime. First, it supports running Lambda functions on AWS Graviton2 processors, offering up to 34% better price-performance compared to functions running on x86_64 processors. Second, it offers a streamlined implementation with a smaller deployment package and faster function invoke path. Finally, this change aligns Go with other languages that also compile to native code such as Rust or C++, which also run on the provided.al2
runtime.
This migration in most cases does not require any code changes. The only changes relate to how you build your deployment package and configure your function. This blog post outlines the steps required to update your build scripts and tooling to use the provided.al2
runtime for your Go functions.
There is a difference in Lambda billing between the go1.x
runtime and the provided.al2
runtime. With the go1.x
runtime, Lambda does not bill for time spent during function initialization (cold start), whereas with the provided.al2
runtime Lambda includes function initialization time in the billed function duration. Since Go functions typically initialize very quickly, and since Lambda reduces the number of initializations by re-using function execution environments for multiple function invokes, in practice the difference in your Lambda bill should be very small.
Compiling for the provided.al2 runtime
In order to run a compiled Go application on Lambda, you must compile your code for Linux. While the go1.x
runtime allows you to use any executable name, the provided.al2
runtime requires you to use bootstrap as the executable name. On macOS and Linux, here’s the simplest form of the build command:
GOARCH=amd64 GOOS=linux go build -o bootstrap main.go
This build command creates a Go binary file called bootstrap compatible with the x86_64 instruction set for Lambda. To compile for AWS Graviton processors, set GOARCH=arm64
in the preceding command.
The final step is to compress this binary into a ZIP file deployment package, ready to deploy to Lambda:
zip myFunction.zip bootstrap
For users compiling on Windows, Go supports compiling for Linux without using a Linux virtual machine or build container. However, Lambda uses POSIX file permissions, which must be set correctly. Lambda provides a helper tool which builds a deployment package that is valid for Lambda—see the Lambda documentation for details. Existing users of this tool should update to the latest version to make sure their build scripts are up-to-date.
Removing the RPC dependency
The go1.x
runtime uses two processes within the Lambda execution environment to route requests to your handler function. The first process, which is included in the runtime, retrieves function invocation requests from the Lambda runtime API, and uses RPC to pass the invoke to the second process. This second process runs the executable which you deploy, and comprises the aws-lambda-go package and your function code. The aws-lambda-go
package receives the RPC request and executes your function.
The following runtime architecture diagram for the go1.x
runtime shows the runtime client process calling the runtime API to retrieve a function invocation and using RPC to call a separate process containing the function code.
Go functions deployed to the provided.al2
runtime use a simpler, single-process architecture. When building the Go executable, you include the same aws-lambda-go
package as before. However, in this case the aws-lambda-go
package acts as the runtime client, retrieving invocation requests from the runtime API directly.
The following runtime architecture diagram shows a Go function running on the provided.al2
runtime. A single process retrieves the function invocation from the runtime API and executes the function code.
Removing the additional process and RPC hop streamlines the function execution path, resulting in faster invokes. You can also remove the RPC component from the aws-lambda-go
package, giving a smaller binary size and faster code loading during cold starts. To remove the RPC dependency, add the lambda.norpc
tag to your build command:
GOARCH=amd64 GOOS=linux go build -tags lambda.norpc -o bootstrap main.go
Creating a new Lambda function
Once your deployment package is ready, you can create a new Lambda function using the provided.al2
runtime using the Lambda console:
Migrating existing functions
If you have existing Lambda functions that use the go1.x
runtime, you can migrate these functions by following these steps:
- Recompile your binary using the preceding commands, making sure to name your binary bootstrap.
- If you are using the same instruction set architecture, open the runtime settings and switch the runtime to “Provide your own bootstrap on Amazon Linux 2”.
- Upload the new version of your binary as a zip file.
Note: The handler value is not used by the provided.al2
runtime, nor the aws-lambda-go
library, and may be set to any value. We recommend the setting the value to bootstrap
to help with migrating between go1.x
and provided.al2
.
To switch instruction set architecture to Graviton (arm64), save your changes, and then re-open the runtimes settings to make the architecture change.
Migrating Go1.x Lambda container images
Lambda allows you to run your Go code as a container image. Customers that are using the go1.x base image for Lambda containers must migrate to the provided.al2 base image. The Lambda documentation includes instructions on how to build and deploy Go functions using the provided.al2 base image.
The following Dockerfile uses a two-stage build to avoid unnecessary layers and files in your final image. The first stage of the process builds the application. This stage installs Go, downloads the dependencies for the code, and compiles the binary. The second stage copies the executable to a new container without the dependencies of the build process.
- Create a Dockerfile in your project directory:
FROM public.ecr.aws/lambda/provided:al2 as build # install compiler RUN yum install -y golang RUN go env -w GOPROXY=direct # cache dependencies ADD go.mod go.sum ./ RUN go mod download # build ADD . . RUN go build -tags lambda.norpc -o /main # copy artifacts to a clean image FROM public.ecr.aws/lambda/provided:al2 COPY --from=build /main /main ENTRYPOINT [ "/main" ]
- Build your Docker image with the Docker build command:
docker build -t hello-world .
- Authenticate the Docker CLI to your Amazon ECR registry:
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com
- Tag and push your image to the Amazon ECR registry:
docker tag hello-world:latest 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/hello-world:latest
You can now create or update your Go Lambda function to use the new container image.
Changes to tooling
To migrate your functions from the go1.x
runtime to the provided.al2
runtime, you must make configuration changes to your build scripts or CI/CD configurations. Here are some common examples.
Makefiles and build scripts
If you use Makefiles or custom build scripts to build Go functions, you must modify to ensure the executable file is named bootstrap when deploying to the provided.al2
runtime.
Here is an example Makefile which compiles the main.go file into an executable called bootstrap in the bin folder. It also creates a zip file, which you can deploy to Lambda using the console or via the AWS CLI.
GOARCH=arm64 GOOS=linux go build -tags lambda.norpc -o ./bin/bootstrap
(cd bin && zip -FS bootstrap.zip bootstrap)
CloudFormation
If you deploy your Lambda functions using AWS CloudFormation templates, change the Handler
and Runtime
settings under Properties:
Resources:
Function:
Type: AWS::Serverless::Function
Properties:
Handler: bootstrap
Runtime: provided.al2
... # Other required properties
AWS Serverless Application Model
If you use the AWS Serverless Application Model (AWS SAM) to build and deploy your Go functions, make the same changes to the Handler
and Runtime
settings as for CloudFormation. You must also add the BuildMethod
:
Resources:
HelloWorldFunction:
Type: AWS::Serverless::Function
Metadata:
BuildMethod: go1.x
Properties:
CodeUri: hello-world/ # folder where your main program resides
Handler: bootstrap
Runtime: provided.al2
Architectures:
- x86_64
Cloud Development Kit (CDK)
If you use the AWS Cloud Development Kit (AWS CDK), you can compile your Go executable and place it under a folder in your project. Next, specify the location by using awslambda.Code_FromAsset
, and AWS CDK packages the binary into a zip file and uploads it.
// Go CDK
awslambda.NewFunction(stack, jsii.String("HelloHandler"), &awslambda.FunctionProps{
Code: awslambda.Code_FromAsset(jsii.String("lambda"), nil), //folder where bootstrap executable is located
Runtime: awslambda.Runtime_PROVIDED_AL2(),
Handler: jsii.String("bootstrap"), // Handler named bootstrap
Architecture: awslambda.Architecture_ARM_64(),
})
Taking this further, AWS CDK can perform build commands as part of your AWS CDK build process by using the native AWS CDK bundling functionality. With the bundling parameter, AWS CDK can perform steps before staging the files in the cloud assembly. Instead of placing the binary file, place the Go code in a folder and use the Bundling option to compile the code in a Docker container.
This example uses the golang:1.20.1
Docker image. After compilation, AWS CDK creates a zip file with the binary and creates the Lambda function:
// Go CDK
awslambda.NewFunction(stack, jsii.String("HelloHandler"), &awslambda.FunctionProps{
Code: awslambda.Code_FromAsset(jsii.String("go-lambda"), &awss3assets.AssetOptions{
Bundling: &awscdk.BundlingOptions{
Image: awscdk.DockerImage_FromRegistry(jsii.String("golang:1.20.1")),
Command: &[]*string{
jsii.String("bash"),
jsii.String("-c"),
jsii.String("GOCACHE=/tmp go mod tidy && GOCACHE=/tmp GOARCH=arm64 GOOS=linux go build -tags lambda.norpc -o /asset-output/bootstrap"),
},
},
}),
Runtime: awslambda.Runtime_PROVIDED_AL2(),
Handler: jsii.String("bootstrap"),
Architecture: awslambda.Architecture_ARM_64(),
})
If, after upgrading, you receive a Phase: init Status: timeout
error, upgrade the aws-lambda-go
and aws-sdk-go
packages.
Conclusion
Lambda is deprecating the go1.x
runtime in line with Amazon Linux 1 end-of-life, scheduled for December 31, 2023. Customers using Go with Lambda should migrate their functions to the provided.al2
runtime. Benefits include support for AWS Graviton2 processors with better price-performance, and a streamlined invoke path with faster performance.
For more serverless learning resources, visit Serverless Land.