[text]
This Guidance demonstrates how to process remote sensing imagery using machine learning models that automatically detect and identify objects collected from satellites, unmanned aerial vehicles, and other remote sensing devices. Satellite images are often significantly larger than standard media files. This Guidance deploys highly scalable and available image processing services that support images of this size. These services collect, process, and analyze the images efficiently, giving you more time to assess and respond to what you discovered in your imagery.
Please note: [Disclaimer]
Architecture Diagram
Prerequisite:
Design, train, and put model containers into the Model Repository with Amazon Elastic Container Registry (Amazon ECR).
Step 1
Submit an image processing request to the Image Request Queue with Amazon Simple Queue Service (Amazon SQS). You can poll the status of your image request from the Amazon Simple Notification Service (Amazon SNS) topic image or Region status.
Step 2
Model Runner Amazon Elastic Container Service (ECS) Task pulls image processing request from the Image Request Queue using Amazon SQS.
Step 3
Model Runner ECS Task queues image Regions into the Region Request Queue with Amazon SQS.
Step 4
Model Runner ECS Task generates image tiles, invokes models, and stores detections in Amazon DynamoDB State Tables.
Step 5
Model Runner ECS Task aggregates, encodes, and outputs results to an Amazon Simple Storage Service (Amazon S3) bucket or an Amazon Kinesis stream.
Step 6
You can consume the aggregated results for analysis, visualization, and more.
Prerequisite:
Design, train, and put model containers into the Model Repository with Amazon Elastic Container Registry (Amazon ECR).
Step 1
Submit an image processing request to the Image Request Queue with Amazon Simple Queue Service (Amazon SQS). You can poll the status of your image request from the Amazon Simple Notification Service (Amazon SNS) topic image or Region status.
Step 2
Model Runner Amazon Elastic Container Service (ECS) Task pulls image processing request from the Image Request Queue using Amazon SQS.
Step 3
Model Runner ECS Task queues image Regions into the Region Request Queue with Amazon SQS.
Step 4
Model Runner ECS Task generates image tiles, invokes models, and stores detections in Amazon DynamoDB State Tables.
Step 5
Model Runner ECS Task aggregates, encodes, and outputs results to an Amazon Simple Storage Service (Amazon S3) bucket or an Amazon Kinesis stream.
Step 6
You can consume the aggregated results for analysis, visualization, and more.
Well-Architected Pillars
The AWS Well-Architected Framework helps you understand the pros and cons of the decisions you make when building systems in the cloud. The six pillars of the Framework allow you to learn architectural best practices for designing and operating reliable, secure, efficient, cost-effective, and sustainable systems. Using the AWS Well-Architected Tool, available at no charge in the AWS Management Console, you can review your workloads against these best practices by answering a set of questions for each pillar.
The architecture diagram above is an example of a Solution created with Well-Architected best practices in mind. To be fully Well-Architected, you should follow as many Well-Architected best practices as possible.
-
Operational ExcellenceRead the Operational Excellence whitepaper
We developed internal pipelines that allow you to deploy and validate the software in this Guidance across multiple Regions and stages, helping you to integrate and make changes as needed. This, coupled with our team of engineers specifically tasked with managing this Guidance and responding effectively to any incidents or events, ensures you're consistently well-architected.
-
SecurityRead the Security whitepaper
Resources in this Guidance are configured following our Best Practices for Security, Identity & Compliance. It continually undergoes routine security-centric analysis from our own Application Security (AppSec) team.
You should be aware that because this system depends on integration with your existing infrastructure and is not managed through a command line interface (CLI), the Guidance doesn’t specifically address authentication and authorization for people and machine access. Instead, it goes through rigorous reviews by our AppSec team to ensure you are not exposed to vulnerabilities when deploying the services as they are designed.
-
ReliabilityRead the Reliability whitepaper
This Guidance is distributed across various Availability Zones and operates on a multi-node Amazon ECS cluster, managed by AWS Fargate, ensuring you have consistent availability.
Amazon SQS queues and Amazon SNS topics disseminate job metadata for you to monitor. These provide you with the capability to track the status and progression of your submitted jobs. This Guidance is equipped with comprehensive embedded metrics and dashboards that are deployed standard with the services.
A continuous integration/continuous deployment (CI/CD) pipeline is utilized, which consistently constructs, deploys, and authenticates the ongoing releases of this Guidance through internal Amazon procedures. Data persistence and backup can be configured with the AWS Cloud Development Kit (AWS CDK) for all deployed resources. We have incorporated retry policies within our job queues and integrated logic into the code to manage graceful failures or partial completions of job requests, while providing status updates for those requests. Additionally, our Amazon ECS cluster features node failover, enabling other nodes to retry any tasks should a failure occur. Finally, we utilize comprehensive integration and load testing methods to confirm the performance and dependability of this Guidance under a wide range of scenarios.
There are constraints you need to be aware of that may affect reliability. When working with extremely slow models against extensive sets of imagery, some calibration is necessary to ensure the optimal performance of this Guidance. This includes managing the models, imagery size, training data, and data sets. Each of these variables dictates how well the model will run.
-
Performance EfficiencyRead the Performance Efficiency whitepaper
Default settings are in place to illustrate the basic capabilities for you. These policies enable the service to dynamically scale up or down, according to the influx and speed of incoming job demands. To meet varying workload requirements, from scaling to traffic, and data access patterns, we employ autoscaling constructs within the AWS CDK framework. This allows you to define and manage the application's autoscaling patterns. Importantly, these configurations and options are made readily accessible to you.
The location of this Guidance can be selected to decrease latency and improve performance by always running from within a single virtual private cloud (VPC) infrastructure.
-
Cost OptimizationRead the Cost Optimization whitepaper
This Guidance assesses cost by analyzing the resources integral to the architectural framework. By employing load testing, repeated iterations, and data collection, we've chosen our services, along with their resource allotments, according to the demonstrated requirements from what we've heard from our customers.
We determine cost based on performance needs and provide the option for you to tailor the service to accommodate your unique mission objectives. This flexibility empowers you to opt for more dynamic scaling policies and larger instance types to enhance performance at a higher cost, or alternatively, to choose less aggressive scaling for cost savings.
-
SustainabilityRead the Sustainability whitepaper
Through the use of autoscaling constructs within AWS CDK, we dynamically adjust to match load requirements and ensure efficient resource allocation, thus ensuring that only the necessary minimum resources are deployed at any given time in the Guidance’s framework. All data stores are configured with best practices and patterns abstracted through constructs in AWS CDK.
For demonstrative or "test drive" use cases, this Guidance is designed to operate efficiently with minimal allocation of resources, thus reducing the amount of required hardware for provisioning. This is the only instance in which hardware is needed.
Implementation Resources
The sample code is a starting point. It is industry validated, prescriptive but not definitive, and a peek under the hood to help you begin.
Related Content
Processing Overhead Imagery Against Computer Vision Models
This sample code orchestrates queue monitoring, tile decomposition, per-tile model invocation, and result aggregation for processing high-resolution satellite images.
Tiling Overhead Imagery on AWS
This sample code efficiently serves satellite imagery tiles for dynamic, high-resolution geographical data visualization.
Disclaimer
The sample code; software libraries; command line tools; proofs of concept; templates; or other related technology (including any of the foregoing that are provided by our personnel) is provided to you as AWS Content under the AWS Customer Agreement, or the relevant written agreement between you and AWS (whichever applies). You should not use this AWS Content in your production accounts, or on production or other critical data. You are responsible for testing, securing, and optimizing the AWS Content, such as sample code, as appropriate for production grade use based on your specific quality control practices and standards. Deploying AWS Content may incur AWS charges for creating or using AWS chargeable resources, such as running Amazon EC2 instances or using Amazon S3 storage.
References to third-party services or organizations in this Guidance do not imply an endorsement, sponsorship, or affiliation between Amazon or AWS and the third party. Guidance from AWS is a technical starting point, and you can customize your integration with third-party services when you deploy the architecture.