Customer Stories / Aerospace & Satellite

2025

How NASA JPL Processes 70 TB of Satellite Data Products a Day Using Amazon EC2 Auto Scaling with Spot Instances

Learn how NASA’s Jet Propulsion Laboratory processes satellite data to create unique maps of Earth’s surface using Amazon EC2 Spot Instances.

70 TB

of satellite data products processed daily

Ability to scale

data processing beyond constraints of on-premises infrastructure

Cost-effective scaling

of compute capacity using Amazon EC2 Spot Instances

High resiliency

and fault-tolerant computing at scale using Amazon EC2 Spot Instances

Overview

The National Aeronautics and Space Administration (NASA) and the Indian Space Research Organisation (ISRO) are collaborating on the NASA-ISRO Synthetic Aperture Radar (NISAR) satellite mission, which will use radar imaging to capture data from nearly all of Earth’s land and ice surfaces. NASA and ISRO are providing L-band and S-band radar systems, respectively, each of which is optimized to help the mission observe a wider range of changes than either one alone. NASA’s Jet Propulsion Laboratory (JPL), which manages the US side of the project for NASA, will process global L-band data, while ISRO will process regional S-band and L-band data, pulling the L-band data from the NASA data archive.

JPL is preparing for the NISAR mission, which will generate more data than any previous NASA Earth mission. To make the best use of that large volume of data, JPL decided to use the scale of the cloud on Amazon Web Services (AWS) to process its data and transform it into final data products. It is using Amazon Elastic Compute Cloud (Amazon EC2) Spot Instances—which help run fault-tolerant workloads for up to a 90 percent discount compared to Amazon EC2 On-Demand pricing—and other AWS services to create a scalable, flexible, agile, and highly available solution that can handle data at high volume and meet mission timelines.

Earth in the outer space

Opportunity | Using AWS to Process 70 TB of Satellite Data Products Daily for the NISAR Mission

JPL is a federally funded research and development center and is managed by the California Institute of Technology. The center has more than 40 active space and airborne missions. When it launches in early 2025, NISAR will be one such mission. The mission will observe and measure some of the most complex changes on Earth, using space-borne synthetic aperture radar (SAR) satellite observations to understand how changes in land and ice surfaces contribute to climate change impacts, natural disasters, and gradual Earth surface changes. SAR is an active instrument and can see through clouds, which is especially useful during natural disasters such as hurricanes and floods.

NISAR is a dual-frequency satellite that will provide radar data of nearly every land and ice surface on the planet. That data will help scientists forecast, compare, and assess changes such as the rate of ice melts, deforestation, or changes in land before and after natural disasters such as earthquakes, leading to new insights into the planet’s dynamics.

JPL uses on-premises infrastructure to process the satellite data from other missions, but the volume and frequency of the data generated from the NISAR mission is expected to be around 70 TB per day. To process that data, NISAR anticipated storage and compute needs that exceeded on-premises capacity, along with the need for flexibility to scale up workloads for lower-latency response needs. JPL had used AWS for an earlier mission, so instead of investing in more on-premises infrastructure—which would have been costlier, more time-consuming, and not as flexible—the center decided to adapt the previously used AWS solution to process NISAR data (see figure 1).

nasa-jpl-spot-case-study-figure1

Figure 1. High-level context of the system components

Solution | Gaining Large, Scalable, and Cost-Effective Compute Capacity Using Amazon EC2 Spot Instances to Transform Data

NISAR’s satellite data will be stored in Amazon Simple Storage Service (Amazon S3), which provides object storage built to retrieve any amount of data from anywhere. To trigger workflows to start processing and transforming the data, JPL will use Amazon Simple Queue Service (Amazon SQS)—which offers fully managed queuing for microservices, distributed systems, and serverless applications—and Amazon Simple Notification Service (Amazon SNS), which is a fully managed Pub/Sub service for A2A and A2P messaging. The data transformations will trigger production rules that help determine whether all the complex conditions and requirements to further process the data have been met. Those rules will run in a combination of AWS Lambda—which helps run code without thinking about servers or clusters—for lightweight rules and containers for the more complex production rules.

The rules will be checked and the data processed in distributed compute nodes on Amazon EC2, which provides secure and resizable compute capacity for virtually any workload. Since most of the processing system is fault tolerant, JPL will take advantage of Spot Instances for scaling. Using Spot Instances, JPL has the flexibility of paying for usage per hour or per second without needing to commit to a specific amount of compute usage, making it even more cost-effective. For critical processing, JPL will also use Amazon EC2 On-Demand pricing, paying for compute capacity by the hour or second with no long-term commitments and On-Demand Capacity Reservations to create and manage reserved capacity on Amazon EC2. The data processing across Spot Instances and On-Demand will be done through automatic scaling fleets that react to the dynamic workload demands.

The data will be processed using algorithms packaged in Docker containers, and the output data will be stored in Amazon S3 buckets. Amazon EC2 provides a variety of compute offerings, such as multi-core CPUs and GPUs, which many of NISAR’s algorithms need for science data processing. The transformed data will be processed to higher and higher levels of derived data products. For example, raw radar signals are decoded, reformatted, and then focused into high-resolution, complex-valued images. These precursor data products are used to generate higher-level data products, such as displacement maps. Once the final data products are generated and stored in Amazon S3, a notification will be sent to the Alaska Satellite Facility Distributed Active Archive Center (ASF DAAC) to ingest the data products into its archive, which is also in the us-west-2 AWS Region. Since the Near Earth Network’s ground stations, the ASF DAAC, and JPL Science Data System are collocated in the same AWS Region, no egress cost will be incurred. After the products reach the DAAC, the relevant JPL teams will be notified through Amazon SNS that the data has been processed (see figure 2). The cycle will then repeat with each downlink pass.

nasa-jpl-spot-case-study-figure2

Figure 2. Use of AWS for processing data and delivering data products to archive centers

This system processes about 4.4 TB of downlinked data daily, generating up to 70 TB of final data products. JPL uses Spot Instances best practices to make sure the algorithms run more optimally on best-sized multi-core Amazon EC2 instances, which helps access more diversified Spot Instances capacity fleets. The system can scale automatically to process data at low latency without significantly increasing compute costs. Automatic scaling provides the required processing capacity when JPL needs it, including for urgent-response scenarios such as natural disasters. Using Spot Instances also increases resiliency, providing JPL with fault-tolerant computing at scale.

Outcome | Making NISAR Data Part of a Free and Open Earth Data Lake

JPL is preparing for NISAR’s launch and is looking forward to making all standard data products generated in AWS available to the scientific community. After launch, NISAR will undergo a 90-day commissioning period before commencing with its main science operational phase, during which it will capture SAR acquisitions of nearly all of Earth’s land and ice surfaces every 6–12 days. The large, scalable, and cost-effective compute capacity provided by Spot Instances will help JPL make this data available quickly as final data products at optimized costs.

Those data products will be archived and distributed from NASA’s Earthdata Cloud, a data lake being built in AWS by DAACs under NASA’s Earth Science Data and Information System Project. NASA’s open science policy will make the archived data—from NISAR and other missions—available to the open scientific community. The consistent and growing collection of data will help scientists develop multiple time series to understand big and subtle changes on the planet, building a better understanding of Earth over time.

About NASA's Jet Propulsion Laboratory

NASA’s Jet Propulsion Laboratory, federally funded by NASA and managed by Caltech, is responsible for planetary robotic spacecraft and Earth observation satellites. It has active space and airborne missions for scientific research.

AWS Services Used

Amazon EC2 Spot Instances

Amazon EC2 Spot Instances let you take advantage of unused EC2 capacity in the AWS cloud and are available at up to a 90% discount compared to On-Demand prices.

Learn more »

Amazon S3

Amazon Simple Storage Service (Amazon S3) is an object storage service offering industry-leading scalability, data availability, security, and performance. Millions of customers of all sizes and industries store, manage, analyze, and protect any amount of data for virtually any use case, such as data lakes, cloud-native applications, and mobile apps.

Learn more »

AWS Lambda

AWS Lambda is a compute service that runs your code in response to events and automatically manages the compute resources, making it the fastest way to turn an idea into a modern, production, serverless applications.

Learn more »

Amazon SQS

Amazon Simple Queue Service (Amazon SQS) lets you send, store, and receive messages between software components at any volume, without losing messages or requiring other services to be available.

Learn more »

More Aerospace & Satellite Customer Stories

Showing results: 1-4
Total results: 37

no items found 

1 10

Get Started

Organizations of all sizes across all industries are transforming their businesses and delivering on their missions every day using AWS. Contact our experts and start your own AWS journey today.