Improve workload sustainability with services and features from re:Invent 2021
At our recent annual AWS re:Invent 2021 conference, we had important announcements regarding sustainability, including the new Sustainability Pillar for AWS Well-Architected Framework and the AWS Customer Carbon Footprint Tool.
In this blog post, I highlight services and features from these announcements to help you design and optimize your AWS workloads from a sustainability perspective.
Architecting for sustainability basics
Environmental sustainability is a shared responsibility between customers and AWS. We maintain sustainability of the cloud by delivering efficient, shared infrastructure. As a customer, you maintain sustainability in the cloud. This means you optimize your workload to efficiently use resources.
The Sustainability Pillar of the Well-Architected Framework provides the following design principles that you can use to optimize your workload for sustainability in the cloud:
- Understand your impact.
- Establish sustainability goals.
- Maximize utilization.
- Anticipate and adopt new, more efficient hardware and software offerings.
- Use managed services.
- Reduce the downstream impact of your cloud workloads.
In the next sections, I share new services and service features that were announced at re:Invent 2021 that are related to these design principles and provide recommendations on how they can help you design and operate your workloads more sustainably.
Understand your impact
Measure the environmental impact of your workloads and propose optimizations to meet your sustainability goals:
- Understand your workload’s environmental impact with new AWS Compute Optimizer resource efficiency metrics and enhanced metrics.
- Optimize scaling decisions and reduce idle resources associated with domain controllers with AWS Directory Service for Microsoft Active Directory and information in Amazon CloudWatch.
- Use the SQL-based metrics query engine to simplify gathering metrics to find optimization opportunities with CloudWatch.
Reduce the total energy required to power your workloads by right-sizing your workloads to ensure high utilization and eliminating or minimizing idle resources.
Optimize your storage
- Use new storage tiers in Amazon Elastic Block Store (Amazon EBS) snapshots, Amazon Timestream, AWS IoT SiteWise, Amazon DynamoDB, and Amazon ElastiCache for Redis to move data to “colder” tiers.
- Support new storage access patterns with the new Amazon Simple Storage Service (Amazon S3) Glacier Instant Retrieval and Amazon S3 Intelligent-Tiering Archive Instant Access
- Delete data that you don’t need by using lifecycle policies with fine-grained controls that use new actions and filters.
- Use storage optimization for AWS Lake Formation to reduce and optimize your storage with its garbage collection and data compaction features.
Optimize data movement
Moving data across networks can contribute to energy consumption and your overall costs. Optimizing networks and data movement can save energy and costs. Optimize the way you access and move data across networks by:
- Use AWS Direct Connect SiteLink to connect an on-premises network directly through the AWS global network. This helps your data travel more efficiently (across the shortest path) rather than using public network.
- Migrate tape backups to the cloud with AWS Snow Family offline tape data migration. This helps you eliminate physical tapes and store your virtual tapes in the cloud with cold storage.
- Automakers that build connected cars generate loads of data. Use AWS IoT FleetWise to reduce the amount of unnecessary data transferred to the cloud.
Optimize your processing
Minimize the resources you use and increase the utilization of resources you use to run your workloads:
- Optimize the resources you use for your machine learning (ML) models with Amazon SageMaker Inference Recommender.
- Reduce AWS Lambda function executions with the new event filtering for Amazon Simple Queue Service (Amazon SQS), DynamoDB, and Amazon Kinesis.
- Minimize the amount of data that needs to be accessed by consumer clusters and provision flexible compute resources in Amazon Redshift with new performance enhancements.
- Use the open-source Kubernetes cluster auto scaling project, AWS Karpenter, to automatically adjust cluster capacity and meet actual demand.
Anticipate and adopt new, more efficient hardware and software offerings
Rapid adoption of new, more efficient technologies helps you reduce the impact of your workloads.
Adopt more efficient instances
- Deploy your workloads on newer, price-performant hardware with the latest Amazon Elastic Compute Cloud (Amazon EC2) instance families from partner-based chips like the Intel-based R6i, AMD EPYC-based M6a, and Mac-based M1
- Use new AWS-built chip instances based in AWS Graviton2 like G5g, Im4gn and Is4gen for more energy-efficient instances.
- Deploy the latest generation of custom-designed AWS Graviton3-based instances: C7g. These instances offer the same performance and use 60% less energy than comparable EC2 instances.
- Gain efficiency with the AWS Trainium ML chip Trn1.
- Use managed services with AWS Graviton2-based chips like AWS Fargate for Amazon Elastic Container Service (Amazon ECS), Amazon Neptune, ElastiCache, and AWS Elastic Beanstalk, which are more energy efficient than comparable EC2 instances.
- Right-size the on-premises AWS workloads that you run with smaller AWS Outposts form factors.
- For workloads that need bare metal instances, use more efficient hardware with the new Amazon EC2 bare metal instances.
Use more efficient software offerings
- Use the next generation of Amazon FSx for Lustre to reduce storage space consumption and make use of more energy-efficient underlying hardware.
- Minimize data movement with pull-through cache repositories in Amazon Elastic Container Registry (Amazon ECR).
- Accelerate deep learning model training by using GPU instances more efficiently with Amazon SageMaker Training Compiler.
- Improve query performance in Amazon Redshift with Automated Materialized Views.
- Reduce self-managed resources with intra-Region peering for AWS Transit Gateway.
- Build event-driven applications based on changes to the data you store in Amazon S3 to be optimized, flexible, and scalable with AWS serverless offerings and use advanced filtering and routing capabilities to send events to 18 targets by integrating Amazon S3 Event Notifications and Amazon EventBridge.
- Use new Amazon S3 Event Notifications to build event-driven applications and optimize resource efficiency of your workloads.
- Simplify operations and retire infrastructure you use to provide remote access to Windows instances with one-click login with AWS Single Sign-On or AWS System Manager Fleet Manager console-based access.
- Decrease SMS-related usage when you test your campaigns in Amazon Pinpoint with the SMS simulator.
Use managed services
We launched several managed services that shift the responsibility of sustainability optimization to AWS:
- Analytics: Amazon RedShift Serverless, Amazon Managed Streaming for Apache Kafka (Amazon MSK) Serverless, Amazon EMR Serverless, and Kinesis Data Streams On-Demand. AWS managed services are scalable, have high average utilization of resources, and are optimized for sustainability.
- ML: Amazon SageMaker Serverless Inference is ideal for applications with intermittent or unpredictable traffic.
- Storage: Amazon FSx for OpenZFS is a fully managed file system that replaces ZFS and other on-premises Linux-based file servers.
- End-user compute: Amazon Workspaces Web and Amazon Appstream2 Elastic (serverless) fleets help reduce idle resources.
- Managed backup options: AWS Backup for Amazon S3 and DynamoDB automate backup retention and deletion.
- Build and manage your wide area network (WAN) with AWS Cloud WAN.
- Migrate and modernize your workload with AWS Mainframe Modernization, which helps move on-premises mainframe workloads to a managed and highly available runtime environment on AWS.
- Create digital twins of real-world systems with AWS IoT TwinMaker.
- Publish and subscribe to third-party APIs using a managed service with AWS Data Exchange for APIs.
Reduce the downstream impact of your cloud workloads
- Use the AWS Rust SDK for the native energy efficiency gains of the Rust programming language.
- Use Amazon CloudWatch RUM to understand the performance of your application and use that information to make it more efficient.
- Review your carbon emissions with the new AWS Customer Carbon Footprint Tool. This helps you define your sustainability key performance indicators (KPIs), optimize your workloads for sustainability, and improve your KPIs.
Having a well-architected, sustainable workload is a continuous process. This blog post brings you AWS announcements from the sustainability perspective. I encourage you to review your workload and find out which of these announcements can be adopted in your workload.
Ready to get started? I encourage you to check out on our What’s New blog for announcements and check them under a sustainability point of view, to identify if they help you improve and meet your sustainability goals.
- Watch the AWS re:Invent 2021 Breakout Sessions – Sustainability
- Other blog posts on architecting for sustainability
- AWS Architecture Monthly magazine, Sustainability issue, August 2021