Containers

High-quality cost-effective AI image generation on Amazon ECS

AI image generation in production has three problems: it’s slow, it’s expensive, and quality depends on serving configuration. A team building a content pipeline or creative tool hits all three at once. Users stare at spinners, GPU bills climb with traffic, and outputs look different depending on which replica handled the request.

At AWS Professional Services, we built a solution that generates high-quality images at low cost by running Z-Image-Turbo, a 6-billion-parameter diffusion transformer, on AWS Inferentia2 accelerators. The service runs on Amazon Elastic Container Service (Amazon ECS) as a standard containerized microservice. It uses three ECS services (one per resolution), each backed by an internal Application Load Balancer (ALB). Compiled model artifacts are cached on Amazon Elastic File System (Amazon EFS) so containers start fast and scale out without recompilation.

This post covers the cost problem, how we solved it with ECS, and how to deploy the solution.

The cost problem

A diffusion model generates an image by starting from random noise and refining it over a series of denoising steps. Each step is one full forward pass through the model. Z-Image-Turbo needs 8 steps where most models need 20–50, but each step still requires the full 6 billion parameters to be resident in accelerator memory. That means:

  • The model consumes 11.5 GB for weights alone, plus 1.6–4.1 GB of working memory during inference.
  • One inference saturates the accelerator. There is no spare capacity to batch requests or run a second one in parallel.
  • The hardware is either generating an image or idle. Utilization drives cost-per-image directly.

At $1.97/hr for an inf2.8xlarge instance, a 512px image (4.87s) costs $0.003 and a 1024px image (19.39s) costs $0.011. Those numbers are low enough for an embedded feature, but only if containers start fast. The bottleneck is model compilation.

These cost estimates are accurate as of publication. Verify current pricing for your AWS Region.

Solution overview

In this post, we walk through deploying our sample inference architecture in your AWS account. The solution is composed of six AWS Cloud Development Kit (AWS CDK) stacks that deploy in dependency order:

  1. Solution: Amazon Virtual Private Cloud (Amazon VPC), AWS Key Management Service (AWS KMS) keys, Amazon EFS file system, and VPC endpoints.
  2. Continuous integration and continuous delivery (CI/CD): Amazon Elastic Container Registry (Amazon ECR) repositories, AWS CodePipeline, AWS CodeBuild projects.
  3. Inference Infra: Amazon Simple Storage Service (Amazon S3) model buckets, Amazon DynamoDB cache table.
  4. Compile SFN: AWS Step Functions state machine, AWS Batch compute environments.
  5. Amazon ECS services: Three ECS services (one per resolution) and an internal ALB.
  6. Web: Amazon CloudFront, Amazon Cognito, AWS Lambda (REST API and WebSocket handler).

Architecture diagram

The following architecture diagram shows how a prompt flows from the React frontend through API Gateway and Lambda to the Inferentia2-backed ECS services.

Architecture diagram showing a prompt flowing from the React frontend through Amazon CloudFront, API Gateway, and Lambda to the Inferentia2-backed Amazon ECS services behind an internal ALB

Figure 1: Request flow from the React frontend through API Gateway and Lambda to the Inferentia2-backed Amazon ECS services

The ECS tasks in this architecture run on Amazon Elastic Compute Cloud (Amazon EC2) inf2.8xlarge instances in an Auto Scaling group. Amazon ECS Managed Instances now supports Inferentia2 and is a great fit for this workload. AWS handles provisioning, patching, and scaling of the underlying instances, so teams can focus on the application.

The Image Generation on Amazon ECS with AWS Inferentia2 code repository is available on GitHub.

Request flow

  1. User enters a prompt in the React frontend (served through Amazon CloudFront).
  2. CloudFront terminates TLS and forwards the WebSocket upgrade to Amazon API Gateway.
  3. API Gateway validates the Cognito JSON Web Token (JWT) and routes to the Lambda handler.
  4. Lambda selects the target resolution and calls the internal ALB’s /invocations endpoint with an X-Target-Resolution header (512, 768, or 1024).
  5. The ALB routes to the matching ECS service’s target group based on that header.
  6. The ECS container encodes the prompt, runs 8 denoising steps, and streams intermediate JPEG frames back.
  7. Lambda relays each frame as a WebSocket binary message to the client.

The round trip from prompt submission to final image takes 5–20 seconds depending on resolution.

The inference workload

A diffusion model generates an image by starting from random noise and refining it over a series of denoising steps, where each step is one full forward pass through the model. Fewer steps means faster generation. This is why the container can stream the image back as it sharpens: each step produces a slightly cleaner frame.

Z-Image-Turbo

The model is a 6-billion-parameter diffusion transformer from the Tongyi-MAI team at Alibaba. It needs only 8 denoising steps where most diffusion models need 20–50, runs a single forward pass per step (many models run two), and produces deterministic output for a given seed.

The architecture, called S3-DiT (Scalable Single-Stream DiT), processes text and image tokens in a unified stream through 30 transformer layers. From an ECS perspective, the key properties are:

  • Memory footprint: 11.5 GB for model weights, plus 1.6–4.1 GB of working memory during inference depending on resolution. That consumes most of the 16 GB high bandwidth memory (HBM) ceiling.
  • Deterministic: Same input produces the same output, byte-for-byte, across instances and restarts. This is why any instance can serve requests interchangeably.
  • Single-threaded: One inference occupies the full accelerator, so an instance is either busy or idle. That is the property the load balancer and auto-scaler are built around.

Figure 2: Denoising progression of Z-Image-Turbo on AWS Inferentia2, refining random noise into a sharp image over 8 steps.

Why Inferentia2

AWS Inferentia2 is Amazon’s purpose-built machine learning (ML) inference accelerator. Each inf2.8xlarge instance ($1.97/hr on demand in us-east-1) has one chip with two NeuronCores, each with 16 GB of HBM. The chip is optimized for attention-heavy transformer workloads, and On-Demand Inf2 capacity is readily available in supported regions.

Our solution: compile once, cache the artifacts on Amazon EFS, and mount them into every container at startup. A container goes from launch to serving traffic in about 2 minutes instead of 30+. This is the core architecture pattern that makes the system practical.

Compile in CI/CD, cache on EFS

In our architecture, an AWS Step Functions state machine orchestrates compilation during the CI/CD pipeline:

  1. The check_artifacts AWS Lambda function inspects EFS for cached Neuron Executable File Format (NEFF) files, comparing a SHA-256 hash of graph-affecting source files.
  2. If the cache is stale, AWS Batch jobs on inf2.8xlarge compile each resolution and write the NEFFs to EFS.
  3. The cache_purge Lambda function removes outdated artifacts.

The SHA-256 hash means that code changes to HTTP handlers, logging, or configuration do not trigger recompilation. Only changes to the model definition, pipeline code, or tensor shapes produce a different hash and force a recompile. This saves 25+ minutes on most deployments.

Container startup with EFS

With compiled artifacts already on EFS, a container starts fast:

  1. Amazon ECS launches the instance and pulls the container image from Amazon ECR.
  2. Container mounts EFS.
  3. Loads pre-compiled NEFF artifacts from EFS into accelerator memory.
  4. Runs a single warmup inference to prime the runtime.
  5. Health check passes (GET /ping returns 200), the ALB registers the target.

Scaling out

Because the compiled artifacts live on EFS (not baked into the container image), scaling out is straightforward. A new container mounts the same EFS volume, loads the same cached NEFFs, and in our testing, started serving in about 2 minutes. There is no per-instance compilation, no model download from Amazon S3, and no cold-compile penalty. The EFS cache turns the scale-out problem into a standard container-scheduling problem.

ECS service architecture

The service runs as three independent deployments, one per resolution. The following sections explain why they are separated and how each is configured.

Why three separate services

Each resolution has a different memory profile and inference latency:

Resolution Memory per container Inference time Requests/hr/instance
512×512 13.1 GB 4.87 s ~739
768×768 15.2 GB 9.81 s ~367
1024×1024 15.6 GB 19.39 s ~186

A single multi-resolution service would need to swap compiled model artifacts per request while managing memory within 1 GB of the 16 GB hardware limit. Separate services let each resolution scale independently. In practice, 512px traffic is typically 3–5x higher volume than 1024px.

Service configuration

Each ECS service runs on inf2.8xlarge instances:

  • Task definition: Single container per task, awsvpc networking mode.
  • Instance type: inf2.8xlarge (1 AWS Inferentia2 chip, 2 NeuronCores, 32 GB HBM).
  • Concurrency: 1 request at a time per instance (the model saturates the accelerator, so no batching is possible).
  • Health check: GET /ping returns 200 only after the model is loaded and warm.
  • Deregistration delay: 120 seconds (allows in-flight inference to complete before draining).
  • Amazon EFS mount: Read-only access point for compiled model artifacts.

Prerequisites

  1. AWS account with service quotas:
    • Running On-Demand Inf instances >= 1 inf2.8xlarge per resolution you want to serve.
    • Request a quota increase through the Service Quotas console if your account has 0 (new accounts typically start at 0 for Inf instances).
    • Standard Amazon VPC, Amazon ECS, and AWS Lambda quotas (defaults are sufficient).
  2. AWS Cloud Development Kit (AWS CDK) Bootstrap: Your account/region must be CDK-bootstrapped:
    npx cdk bootstrap aws://ACCOUNT_ID/REGION
  3. Local tools: Python 3.13+, uv >= 0.4, Docker, Node.js, AWS Command Line Interface (AWS CLI) v2.

Deploy the solution

Configure your environment, then deploy the six stacks in dependency order.

Configure environment

CDK_DEFAULT_ACCOUNT=123456789012
CDK_DEFAULT_REGION=us-east-1

Setup

# Clone and install
git clone https://github.com/aws-samples/sample-image-generation-on-ecs.git && cd sample-image-generation-on-ecs
uv venv --python 3.13 && source .venv/bin/activate
uv sync --group dev

Configure environment variables

ZIMAGE_ENV=dev

Copy the example file and fill in your values:

cp .env.example .env

Deploy

# Deploy solution infrastructure
make bootstrap
# Deploy inference pipeline (includes model compilation + ECS services)
make deploy-inference

make bootstrap deploys the Amazon VPC, AWS KMS keys, Amazon EFS, and Amazon ECR repositories, and the AWS CodePipeline definitions (~5 minutes).

make deploy-inference uploads source to Amazon S3 and triggers the inference pipeline. First deployment takes ~45 minutes (includes model compilation). Subsequent deploys that don’t touch model code take ~20 minutes.

Deploy the web application

For authenticated public access with a UI, gallery, and admin panel:

make deploy-web
make create-admin EMAIL=you@example.com

Generate images

You can generate images through the web UI or directly through the REST and WebSocket APIs.

Web UI

Z-Image web UI with a prompt box, Enhance Prompt option, resolution selector, and Generate button

Figure 3: The Z-Image web UI, where you select a resolution, enter a prompt, and watch frames stream as each step completes

The solution ships a React frontend served through Amazon CloudFront. Select a resolution, enter a prompt, and watch intermediate frames stream as each denoising step completes.

REST API (web stack)

curl -X POST https://{cloudfront-url}/api/generate \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer ${TOKEN}" \
  -d '{
    "prompt": "A campfire is burning in a remote wilderness area under the night sky...",
    "width": 1024,
    "height": 1024,
    "seed": 996776
  }'

WebSocket streaming

Connect to wss://{api-id}.execute-api.{region}.amazonaws.com/v1 for real-time intermediate frames during generation. Each denoising step produces a progressively cleaner image, streamed to the client as a binary WebSocket message.

Generated images

The following images were generated with this architecture. The exact prompt used to generate each image appears with it.

All images were generated on Amazon ECS (inf2.8xlarge) with 8 denoising steps. Given the same prompt, seed, and resolution, the service produced identical output in our testing.

Album art (1024×1024)

Generated album cover art: a red electric guitar engulfed in blue and purple flames in a dark void with shattered glass fragments

Figure 4: Album art (1024×1024) generated with Z-Image-Turbo from the following prompt

Rock band album cover art featuring a red electric guitar engulfed in blue and purple flames, floating in a dark void, surrounded by shattered glass fragments with sharp edges and iridescent surface reflections under dim ambient lighting, bold creative stylized grunge text “IRON ECHO” centered at the top in cracked metallic chrome font with a yellow lightning bolt extending to the left of the text, smaller neon purple text “SHATTERED LIGHT” placed horizontally at the bottom of the composition, fonts with jagged edges and glowing outlines, black background with subtle texture resembling brushed steel, 50mm macro lens perspective emphasizing detailed textures of metal, glass, and flame surfaces, high definition rendering with emphasis on realistic lighting effects and sharp depth of field.

Campfire (1024×1024)

Generated image of a campfire under the Milky Way in a pine forest at night

Figure 5: Campfire (1024×1024) generated with Z-Image-Turbo from the following prompt

A campfire is burning in a remote wilderness area under the night sky. The fire is composed of flame and glowing embers, situated on a flat forest floor surrounded by pine trees. Above the fire, the sky is clear and dark, with the milky way stretching from the horizon into the upper sky. The milky way appears as a bright, swirling band of stars with subtle blue and orange tones. The surrounding trees are tall, with conical shapes, their trunks are dark and textured. Smoke from the fire rises vertically and dissipates into the sky. The atmosphere is cool and calm, with no wind. The scene is illuminated by the soft glow of the fire and the faint light of the stars. The color palette is a combination of warm orange and yellow from the fire and cool blue from the night sky.

The additional samples are available on the Image Generation on Amazon ECS with AWS Inferentia2 GitHub repository.

Cost analysis

All costs use inf2.8xlarge On-Demand pricing at $1.97/hr in us-east-1. Check the Amazon EC2 Inf2 pricing page for current rates.

Resolution Cost per image Latency Throughput Cost per 1,000
512×512 $0.003 4.87 s ~739 images/hr $2.66
768×768 $0.005 9.81 s ~367 images/hr $5.36
1024×1024 $0.011 19.39 s ~186 images/hr $10.59

Monthly cost depends on utilization. An always-on single instance runs ~$1,416/month. Scaling to zero removes Inf2 instance charges when idle. You still pay for other resources such as Amazon EFS, the ALB, VPC endpoints and Amazon CloudFront. See the pricing pages.

Clean up

make destroy

Destroys all 6 stacks in reverse dependency order (~10 minutes). Removes VPC, ECS cluster, ALB, EFS, ECR, S3 buckets, Cognito, CloudFront, and CodePipeline.

Conclusion

The cost problem with production image generation comes down to two things: expensive compilation and slow cold starts. The compile-once, cache-on-Amazon EFS pattern helps address both. The CI/CD pipeline compiles the model (25–45 minutes, paid once), and every container mounts the cached artifacts at startup (2 minutes to serving). The result is $0.003–$0.011 per image on inf2.8xlarge, which is low enough that image generation works as an embedded feature rather than a standalone service.

Feeling ready to build your own image generation service? Begin your journey with the Image generation on ECS quick start instructions!


About the authors

Nick Biso

Nick Biso

Nick Biso is a Machine Learning Engineer at AWS Professional Services. He solves complex organizational and technical challenges using data science and engineering. In addition, he builds and deploys AI/ML models on the AWS Cloud. His passion extends to his proclivity for travel and diverse cultural experiences.

Srividya Ponnada

Srividya Ponnada

Srividya is an Associate Delivery Consultant at AWS Professional Services. She helps customers take machine learning (ML) solutions from data exploration to production at scale. She focuses on making applied and generative AI practical in production, balancing quality, cost, and throughput so teams can deploy with confidence. Her approach is grounded in research, with peer-reviewed publications spanning perception, decision-making for autonomous systems and adversarial robustness, reliability in production ML systems. She holds a Master’s in Computer Science from the University of Maryland, College Park.

Maria Masood

Maria Masood

Maria Masood specializes in agentic AI, reinforcement fine-tuning, and multi-turn agent training. She has expertise in machine learning, spanning large language model customization, reward modeling, and building end-to-end training pipelines for AI agents. A sustainability enthusiast at heart, Maria enjoys gardening and making lattes.