AWS News Blog

AWS Lambda now supports GPU instances for Event Functions on Lambda Managed Instances

Voiced by Polly

GPU support is now available for EC2 G-series GPU instances for Event Functions running on AWS Lambda Managed Instances (LMI). You can now run GPU-accelerated workloads, including machine learning inference, media processing, rendering, and scientific computing, on GPU instances with serverless simplicity. GPU-accelerated functions are deployed as container (Open Container Initiative, OCI) images. You build an OCI image, point the function at it, and let Lambda handle provisioning, scaling, and event integration. There are no GPU drivers, CUDA versions, or AMIs to reconcile, and no clusters or device plugins to operate. GPU-accelerated functions run on the same Lambda Managed Instances foundation that already gives you Amazon EC2 pricing and specialized compute without managing servers.

Why customers need this
Customers keep asking to run GPU workloads with the simplicity of serverless. Building on self-managed GPU infrastructure means selecting instances, installing and reconciling GPU drivers and CUDA libraries, planning capacity, writing scaling logic, and hand-building the event integrations that connect the GPU step to the rest of the application. Serverless GPU services that hide that work often sit outside your AWS environment, with no native event sources, identity, or networking.

Cost is the second problem. GPUs are expensive, and utilization is usually low because a single workload rarely saturates a GPU. A small inference task can leave most of a provisioned GPU idle while you pay for the whole thing.

This feature closes both gaps. You bring a function, and Lambda runs it on a GPU in minutes, triggered by the same AWS events as any other Lambda Event Function, billed and secured the same way. It also maximizes utilization of every GPU you pay for, by consolidating functions onto instances and keeping GPUs busy across concurrent requests.

What you can build
You can run both AI/ML inference and non-ML workloads using GPU-accelerated compute on Lambda Managed Instances. This capability is optimized for inference rather than training, and custom models rather than large foundation models already available on services like Amazon Bedrock. Common workloads include:

  • Custom model inference: decision models (such as Strands Decider), computer vision, speech-to-text and text-to-speech, recommendation, document analysis, and small-to-mid-size language models deployed as part of an application.
  • Agentic and event-driven AI: custom inference triggered directly by AWS events, composed with the rest of the Lambda programming model.
  • Media processing and transcoding: video and image pipelines triggered on upload.
  • 3D rendering and generative media: frame and asset rendering, diffusion, and text-to-video generation.
  • Scientific computing and financial modeling: simulations, matrix computation, and risk analysis.

Part of the Lambda programming model
A GPU-accelerated function is one step in a larger application, and it composes with the rest of Lambda. Lambda durable functions orchestrate multi-step, long-running workflows around the GPU step, with checkpointing and automatic recovery. Lambda MicroVMs run untrusted or model-generated code in isolated, stateful sandboxes. In an agentic workload, a MicroVM executes the tool-use logic on CPU, then calls a GPU-accelerated function running on Lambda Managed Instances for inference.

GPU-accelerated functions also support the extended function timeout on Lambda Managed Instances, up to 90 minutes for asynchronous and event source mapping invocations, so you can run batch and other long-running GPU jobs, such as media transcoding or multi-step inference, without re-architecting. GPU-accelerated functions support container images up to 32 GB (10 GB per file), up to 32 GB of ephemeral storage, 2,048 to 32,768 MB of memory, 3 to 48 GB of accelerator memory, up to 48 concurrent requests per execution environment, and up to 15,000 vCPUs per capacity provider.

Let’s try it out
To show what this looks like in practice, I deployed an open-source large language model (LLM) from Hugging Face as a Lambda function and sent it a prompt. With the vLLM base image from Lambda, I only set the Hugging Face model ID in the MODEL_ID environment variable, without writing any inference code. I used Qwen/Qwen2.5-1.5B-Instruct, a small instruction-tuned model that suits in-application tasks such as summarization, classification, and extraction. The flow has four steps: create a capacity provider, build and push the container image, create a function from that image, then publish and invoke. Before you begin, request an increase to your G-series instance quota (listed as Running On-Demand G and VT instances), because new accounts might start with a quota of 0. For more information, see On-Demand Instance quotas.

First, I created a capacity provider, where I specify the GPU instance types Lambda can use and let it auto-scale. Listing several instance types lets Lambda pick the most cost-effective one that is available and fall back across generations when capacity is tight.

{
    "CapacityProviderName": "lmi-gpu-cp",
    "VpcConfig": {
        "SubnetIds": ["subnet-aaaa", "subnet-bbbb", "subnet-cccc"],
        "SecurityGroupIds": ["sg-aaaa"]
    },
    "PermissionsConfig": {
        "CapacityProviderOperatorRoleArn": "arn:aws:iam::111122223333:role/lmi-operator-role"
    },
    "InstanceRequirements": {
        "AllowedInstanceTypes": [
            "g5.2xlarge", "g6.2xlarge", "g6e.2xlarge"
        ]
    },
    "CapacityProviderScalingConfig": {
        "ScalingMode": "Auto",
        "MaxVCpuCount": 96
    }
}
aws --region us-east-2 \
  lambda create-capacity-provider \
  --cli-input-json file://cli-skeletons/create-capacity-provider_gpu.json

The Lambda console shows the new capacity provider, lmi-gpu-cp, in the Active state. Note the VPC configuration with its subnets and security group.
Lambda console showing the GPU capacity provider lmi-gpu-cp in the Active state with its VPC configuration

Figure 1: The capacity provider is Active, with its VPC configuration.

Next, I built the container image. Lambda provides a curated vLLM base image that includes the vLLM inference server, PyTorch, Transformers, and the CUDA toolkit, along with a runtime interface client that supports concurrent invocations. The image serves the model for me, so I don’t write a handler or any inference server code. My entire Dockerfile is one line:

FROM public.ecr.aws/lambda/deep-learning-containers/vllm:cuda-v1

After pushing the image to Amazon ECR, I created the function. I chose the model with the MODEL_ID environment variable, which accepts a Hugging Face or ModelScope repository, an Amazon S3 location, or a local path. Lambda downloads the model, manages GPU memory, and serves requests. Qwen2.5-1.5B-Instruct is a public model, so I didn’t need a Hugging Face token. For private or gated models, or to get higher download rate limits, you can also set HF_TOKEN.

GPU-accelerated functions use AcceleratorConfig to request accelerator memory. AcceleratorMemorySize is the minimum GPU memory the function needs, and Lambda places the execution environment on the smallest available GPU that meets it, so the GPU you get can have more memory than you requested. I set PerExecutionEnvironmentMaxConcurrency to 4, the default. The vLLM server batches concurrent requests on the GPU, so invocations share one copy of the model and the GPU stays busy. I also set ExecutionEnvironmentMemoryGiBPerVCpu to 4, which gives the function the 16,384 MB of memory and 4 vCPUs it needs while leaving capacity on the 2xlarge instance for Lambda’s management components.

{
    "FunctionName": "lmi-gpu-inference",
    "PackageType": "Image",
    "Code": { "ImageUri": "111122223333.dkr.ecr.us-east-2.amazonaws.com/lmi-gpu-inference:latest" },
    "Role": "arn:aws:iam::111122223333:role/lmi-gpu-inference-execution-role",
    "Timeout": 900,
    "MemorySize": 16384,
    "EphemeralStorage": { "Size": 32768 },
    "Environment": { "Variables": { "MODEL_ID": "Qwen/Qwen2.5-1.5B-Instruct" } },
    "CapacityProviderConfig": {
        "LambdaManagedInstancesCapacityProviderConfig": {
            "CapacityProviderArn": "arn:aws:lambda:us-east-2:111122223333:capacity-provider:lmi-gpu-cp",
            "ExecutionEnvironmentMemoryGiBPerVCpu": 4,
            "PerExecutionEnvironmentMaxConcurrency": 4
        }
    },
    "AcceleratorConfig": { "AcceleratorMemorySize": 16 }
}
aws --region us-east-2 \
  lambda create-function \
  --cli-input-json file://cli-skeletons/create-gpu-llm-function.json

The function overview shows what makes this a GPU-accelerated function: the attached capacity provider that supplies the GPU instances and the vLLM container image deployed from Amazon ECR, all wired into the same function model you already use.
Lambda console function overview for lmi-gpu-inference showing the attached capacity provider lmi-gpu-cp, the container image from Amazon ECR, and the x86_64 architecture

Figure 2: The lmi-gpu-inference function overview, showing the attached capacity provider and the vLLM container image deployed from Amazon ECR.

Functions on Lambda Managed Instances run from a published version, so I published one. Publishing provisions the execution environments, and each one downloads the model weights and warms up the engine before the version becomes active. The cold start happens during the publish step, so the first caller doesn’t wait for it.

aws --region us-east-2 \
  lambda publish-version \
  --function-name lmi-gpu-inference \
  --publish-to LATEST_PUBLISHED

Then I sent the model a prompt. The payload is a standard OpenAI request body: a messages array routes to chat completions, and a prompt field routes to text completions.

aws --region us-east-2 \
  lambda invoke \
  --function-name 'lmi-gpu-inference:$LATEST.PUBLISHED' \
  --cli-binary-format raw-in-base64-out \
  --payload '{"messages": [{"role": "user", "content": "Explain serverless GPU inference in one sentence."}], "max_tokens": 128}' \
  /dev/stdout

The model answered from a GPU inside the Lambda execution environment, and the response is a standard OpenAI completion object, so any OpenAI-compatible client works with it.

{
  "model": "Qwen/Qwen2.5-1.5B-Instruct",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Serverless GPU inference is a cloud-based service that allows for efficient execution of machine learning models on NVIDIA GPUs without the need for infrastructure management or maintenance."
      },
      "finish_reason": "stop"
    }
  ],
  "usage": { "prompt_tokens": 39, "completion_tokens": 31, "total_tokens": 70 }
}

Four concurrent invocations finished in about the same time as a single one, on one execution environment that shares one copy of the model in GPU memory. From a Hugging Face model ID to a live, OpenAI-compatible GPU endpoint took about 10 minutes, most of it unattended while Lambda provisioned capacity and warmed up the model.

That’s a working LLM endpoint with no inference server code, no GPU drivers, and no cluster to manage. To try a different model, I update MODEL_ID and publish a new version, with no image rebuild. Each version is an immutable snapshot, so different models can run side by side as separate versions. It is also a Lambda Event Function, so I can trigger it from an Amazon SQS queue to process documents in batches, put Amazon API Gateway in front of it, or call it as the inference step in a Lambda durable functions workflow.

For production, copy the model weights to an S3 bucket in the same Region, point MODEL_ID at that bucket, and set VLLM_LOAD_FORMAT=runai_streamer so the weights stream straight into memory. This gives faster starts and removes the dependency on the Hugging Face Hub. For more guidance, see the best practices for running inference workloads. Beyond LLM inference, you can also run custom ML inference, media processing, and other GPU workloads. See the Lambda documentation for best practices.

The configuration tab shows the settings that matter for a GPU-accelerated function: 16,384 MB of memory, the 15-minute timeout, and 32 GB of ephemeral storage for the downloaded weights, alongside the function-level capacity provider settings: per-environment concurrency of 4, the 4:1 memory-to-vCPU ratio, and 16 GB of accelerator memory.

Lambda console configuration tab for lmi-gpu-inference showing 16,384 MB of memory, the 15-minute timeout, 32 GB of ephemeral storage, and the function-level capacity provider settings including per-environment concurrency, the memory-to-vCPU ratio, and accelerator memory

Figure 3: The function’s configuration, including memory, timeout, and 32 GB of ephemeral storage, and the function-level capacity provider settings: per-environment concurrency, the memory-to-vCPU ratio, and accelerator memory.

When you finish testing, delete the function and then the capacity provider so that Lambda stops the GPU instances and you stop paying for them.

How it works
The capabilities below are what you get out of the box, each one removing GPU work you would otherwise own.

Skip the GPU software setup. Curated base images ship with the frameworks and tools handled for you, so you do not install or reconcile them: base:cuda (CUDA toolkit and runtime), cupy:cuda (CuPy, NumPy, SciPy, Pandas, Numba), pytorch:cuda (PyTorch, torchvision, torchaudio, transformers, FFmpeg), and serving images vllm:cuda and sglang:cuda. Use them directly, extend them, or bring a fully custom image.

No GPU infrastructure to operate. Lambda handles driver compatibility, provisioning, routing, and load balancing, so you ship functions instead of running GPU fleets. Deploy with the same Console, CLI, AWS CloudFormation, AWS CDK, AWS SAM, and Terraform workflow as any Lambda Event Function.

Trigger GPU work from any AWS event. The same 200+ event sources that trigger Event Functions, including Amazon S3, Amazon SQS, Amazon API Gateway, and Amazon EventBridge, trigger GPU-accelerated functions directly, with the same scaling, error handling, and retries. No custom integration code.

Serve concurrent requests on each GPU. Each execution environment can process multiple concurrent invocations on the same GPU, which you control with the per-environment concurrency setting. Raising it packs more requests onto each GPU instance to drive utilization up, so you get more throughput from the capacity you already pay for.

Multiple instance types. Specify multiple instance types or generations, and Lambda selects the most cost-effective available option and falls back across your list to reduce Insufficient Capacity Errors.

Capacity options. GPU instances run as Amazon EC2 On-Demand instances, and your existing Savings Plans and Reserved Instances apply. For predictable GPU availability, create an EC2 On-Demand Capacity Reservation (ODCR) that matches your capacity provider configuration, and Lambda uses that reserved capacity automatically.

Large container images and storage. Container images up to 32 GB are supported, along with up to 32 GB of NVMe ephemeral storage in /tmp. Seekable OCI (SOCI) lazy loading lets your function begin executing before the full image is downloaded, which reduces execution environment startup time by up to 70%.

See GPU usage without extra setup. GPU utilization, GPU memory utilization, GPU memory used, and GPU power utilization appear alongside standard function metrics in Amazon CloudWatch and in the AWS Lambda console.

After invoking the function, the console surfaces the GPU-specific metrics next to the standard ones. Track GPU utilization, GPU memory utilization, GPU memory used, and GPU power utilization to see how much of the accelerator each invocation actually uses, which is the signal that drives both auto-scaling and cost.

Lambda console Monitor tab showing GPU utilization, GPU memory utilization, GPU memory used, and GPU power utilization charts for a GPU inference function under load

Figure 4: GPU utilization, GPU memory utilization, GPU memory used, and GPU power utilization for the vLLM function under concurrent load.

Pricing and availability
This feature is generally available today, supporting the Amazon EC2 G-series (G4dn, G5, G6, G6f, Gr6, Gr6f, G6e) at launch, with more instance families to come. You pay the underlying EC2 instance fee (eligible for Savings Plans and Reserved Instances), plus a 9.75 percent management fee for GPU instances, and the standard Lambda request fee of $0.20 per million requests. For pricing and Region availability, see AWS Lambda pricing.

You can also build with your AI coding assistants and agents: explore the Builder Center and connect your tools with the Agent Toolkit for AWS, which bundles the AWS MCP Server and curated agent skills.

To get started, visit the AWS Lambda console and see the GPU-accelerated functions documentation.

Daniel Abib

Daniel Abib

Daniel Abib is a senior specialist solutions architect at AWS, focused on generative AI and Amazon Bedrock — and passionate about serverless. He helps startups and enterprises build AI-powered applications and modernize with cloud-native architectures. A three-time Ironman finisher and four-time re:Invent speaker, he brings the same endurance mindset to building cloud solutions.