How Agriculture and Agri-Food Canada built a cloud-based coherence change detection solution with AWS

Monitoring agricultural practices across Canada’s vast landscape is essential for preserving soil health and water quality. To address this challenge at scale, Agriculture and Agri-Food Canada (AAFC) collaborated with Amazon Web Services (AWS) to build a cloud-based proof of concept (PoC) solution that processes synthetic aperture radar (SAR) satellite imagery, which can detect changes in agricultural land use and tillage practices.

With this solution, the research team at AAFC built on the scientific research described in “Dingle Robertson et al., 2023 (Heliyon),” which validated the use of coherent change detection (CCD) techniques to track tillage and harvest events across agricultural fields using Sentinel-1 Single Look Complex (SLC) and RADARSAT Constellation Mission (RCM) satellite data. The solution—which runs on AWS—takes these findings a step further, showcasing how cloud-based architectures can scale scientific workflows into operational systems that support national monitoring and policy objectives.

From science to scalable cloud workflows

The original research demonstrated that CCD values derived from pairs of SAR images can effectively identify changes in the ground surface when farmers harvest, till, or chemically treat a field. The surface roughness and structure alter the radar signal’s coherence, producing detectable patterns. Sentinel-1’s SAR imagery is particularly well suited to this type of research using satellite Earth observation data.

Sentinel-1 satellites collect radar images of Earth by capturing quick snapshots, which are called bursts, while sweeping across different areas, similar to taking multiple overlapping photos to create a panorama. The SLC format preserves the complete radar signal information for each burst, enabling precise scientific analysis of ground conditions—making it ideal for tasks such as measuring ground movement or creating detailed surface maps.

By automating geospatial analytics of Earth observation data in the cloud, AAFC can process large volumes of satellite imagery faster and at lower cost than on research scientists’ own desktop processors—enabling national-level insights into agricultural management activity and processes.

AAFC’s solution focused on the Canadian region surrounding Ontario, with the goal of expanding to pan-Canadian coverage. In future iterations, scientists will apply the scientific CCD algorithm to multiple use cases, including monitoring of conservation tillage practices, mapping of crop residue, and assessment of soil health programs. The following image illustrates Sentinel-1 footprints of data collection across Canada, and highlights the particular area of data collection automated for the POC.

Figure 1: Areas of interest (AOIs) across Canada for Sentinel-1 data. PoC region in Ontario highlighted

Building the CCD pipeline on AWS

AAFC and AWS collaborated to design and deploy the solution in the Shared Services Canada (SSC) Landing Zone Accelerator (LZA), a more secure, compliant cloud environment specifically designed for federal research and development workloads. This environment meets Government of Canada security and data sovereignty requirements.

AAFC’s solution uses containerized scientific applications, event-driven automation, and managed compute services to orchestrate the workflow from data ingestion to product generation.

Architecture overview

The architectural diagram below depicts how this satellite data processing solution was automated.

Figure 2: AWS architecture of the AAFC CCD solution

At a high level, the architecture consists of the following components:

• AWS Lambda (Lambda) – A Lambda function is used to download Sentinel-1 SLC from NASA’s Alaska Satellite Facility Distributed Active Archive Center (ASF DAAC), which provides fast, reliable data delivery of satellite and airborne-acquired SAR data. These satellite data files are downloaded from ASF DAAC as .zip files and stored in an Amazon S3 incoming data bucket.

• Amazon Simple Storage Service (Amazon S3) – Amazon S3 serves as the data lake for incoming Sentinel-1 SAR and RCM imagery and processed results. Amazon S3 is also used to allow config files defining area and time extents to be uploaded by the user. When new data is uploaded, that upload event automatically triggers the processing workflow.

• Amazon DynamoDB (DynamoDB) – A NoSQL table in DynamoDB keeps track of the state of satellite data files that have been downloaded and placed into Amazon S3 buckets based on the location areas of interest (AOIs) and time bounds that the AAFC missions need.

• AWS Step Functions (Step Functions) – Step Functions acts as the orchestration layer, managing the workflow state and invoking containerized tasks on Amazon ECS with AWS Fargate.

• Amazon Elastic Container Service (Amazon ECS) with AWS Fargate – Amazon ECS runs the European Space Agency Sentinel Application Platform (ESA SNAP) Docker container in a fully managed, serverless compute environment. ESA has developed a series of no-cost open source toolboxes for the scientific exploitation of Earth observation missions. SNAP is one of these toolboxes. The ESA toolboxes support the scientific exploitation for the Sentinel 1, 2, and 3 missions, European Remote Sensing/Environmental Satellite (ERS/Envisat) missions, and a range of national and third-party missions. SNAP has been containerized in Docker and is available for public use. Each containerized task processes one or more satellite scenes, generating coherence products using SNAP’s Graph Processing Tool (GPT) commands.

• Amazon Elastic Container Registry (Amazon ECR) – Amazon ECR hosts the ESA SNAP container images.

This modular architecture means that each processing stage is automated, reproducible, monitored, and scalable while remaining compliant with federal science and security requirements. The following image illustrates the algorithm details of Coherence Change Detection.

How the solution works

Figure 3: CCD algorithm details

Step Functions tracks each state transition from imagery download and coherence estimation to terrain correction and export, mirroring the same steps described in the SNAP coherence estimation process. As tasks complete, the final products are published back to Amazon S3 for visualization or downstream analysis. This event-driven pattern provides the foundation for scaling scientific workloads to national levels of processing while maintaining transparency and traceability.

With the system’s modularity and elasticity, it can be run on demand from small-scale tests to full national coverage. Processing time varies from around 15 to 30 minutes per image pair, with approximately 10 concurrent processes running at a time. This enables scalability from small test areas to national-level coverage within only a few hours. The workflow can be run on demand for entire growing seasons, providing flexibility to adapt and refine methods as needed.

Processing a full month of coherence analysis required approximately 2 terabytes (TB) of total storage across 272 scene pairs. The entire computation was completed in almost exactly 12 hours with a total cost of approximately USD $200. This flexibility accelerates research, reduces setup time, and enables iterative experimentation. In the future, an automated data acquisition and preprocessing routine could continuously ingest Sentinel-1 imagery, providing shared access for multiple research teams working across Canada. The following image illustrates the example output data of CCD processing on Sentinel-1 data near Montreal, Canada.

Figure 4: Example output of CCD processing on Sentinel-1 data near Montreal, Canada

Benefits of the cloud-based approach

By moving the processing pipeline to AWS, AAFC can:

• Automate repeatable scientific workflows across large geographies without manual intervention, facilitating validation of algorithmic output by AAFC scientists
• Use containerized applications such as ESA SNAP to enable scientific reproducibility and consistent results
• Integrate nearly real-time triggers using Amazon EventBridge and Step Functions, reducing time-to-insight for field-level changes
• Prepare for scale to extend processing from Ontario across Canada

From PoC to national impact

The successful completion of this PoC solution demonstrates how cloud-based approaches can accelerate scientific research and innovation in Canada’s public sector. By combining the science of CCD with the elasticity and automation of AWS, AAFC now has the foundation to scale environmental monitoring and policy analysis across provinces and seasons.

Next steps could include expanding Step Functions to incorporate automated tillage detection logic, enabling detection of specific agricultural events directly from coherence change. This evolution would extend the existing pipeline to support advanced classification and multisource analysis, further strengthening the capacity for nearly real-time environmental intelligence. Amazon EventBridge will be used to enable event-driven processing. Each time new satellite Earth observation data is uploaded to Amazon S3, Amazon EventBridge initiates a Step Functions workflow. The workflow calls one or more AWS Fargate for Amazon ECS tasks using the RunTask integration pattern, passing metadata about the dataset and requested processing steps.

Future iterations of the solution will explore integrating more SAR missions, expanding coverage nationwide, and incorporating machine learning to classify agricultural practices automatically.

Conclusion

The CCD solution built by AAFC demonstrates how modern cloud architectures can transform complex Earth observation science into scalable, automated workflows. By combining containerized processing with event-driven orchestration, the system enables rapid automated generation of coherence products from Sentinel-1 and RADARSAT data, turning raw satellite Earth observation data into actionable insights for monitoring agricultural activities.

This PoC establishes a reusable foundation for federal research organizations to conduct large-scale geospatial analytics on AWS. It supports repeatable and cost-effective processing, streamlines the integration of new data sources, and provides a path to expand coverage across Canada. With its cloud-based design, the solution is ready to help researchers and policy teams generate timely, evidence-based, data-driven insights that advance sustainable agriculture and environmental stewardship.

To get practical experience with building this kind of automated data management workflow, try the Serverless Land workflows, the Large-scale Data Processing with Step Functions workshop, and the Serverlesspresso workshop.