AWS Case Study: European Space Agency

Data User Elements (DUE) is an important program within the European Space Agency (ESA). The program is conducted at the ESA Centre for Earth Observation in Frascati, Italy. DUE uses satellites to collect valuable data about the current state of the planet. The program is about demonstrating the usefulness of Earth Observation products and services to end users worldwide. DUE provides this data to scientists, governmental agencies, and private organizations around the world. The data is used for a myriad of purposes, including monitoring the environment, improving the accuracy of weather reporting, and assisting disaster relief agencies.

Although much of DUE’s work is done by satellites, some of the program’s data storage and computing infrastructure is built on Amazon Web Services (AWS). DUE uses Amazon’s scalable Web storage, Amazon Simple Storage Service (Amazon S3), to house and retrieve its images and other end-user products. To manage its files within Amazon S3, the program utilizes a third-party tool based in s3fs, a FUSE filesystem.

In addition to Amazon S3, DUE is also using Amazon’s resizable computing environment, Amazon Elastic Compute Cloud (Amazon EC2), to execute a statistical system that examines the program’s service usages. DUE is using the third-party tool AWStat for its log analysis. In the future, the program hopes to expand its use of Amazon EC2 to include other remote sensing applications.

DUE chose AWS because of its economical pay-as-you-go system, as well as its quick start-up. Jose Ramos, an Earth Observation computer systems engineer, says, “The implementation and deployment stages of our complete distribution and statistics system were unexpectedly quick and easy. It's typical to expect delays when procuring and installing new dedicated hardware, but AWS freed us from that worry and worked within our tight schedule.”

DUE also benefits from the easy manner in which features can be scaled and customized with AWS. This ease-of-use prevents the program from having to hire an external entity to manage its infrastructure. Jose Ramos explains, “With AWS, we have a flexible infrastructure storage system for our data that can scale up to as much as we need. In addition, the solutions offered by AWS are simple to use, so we can manage them ourselves.”

The program reports that during peak usage, AWS helps DUE provide images and other products to over 50 thousand users around the world. This usage can equal 30 terabytes of information at one time.

DUE plans to continue its partnership with AWS while it focuses on conducting its important mission to gather data about Earth. Ramos reiterates his enthusiasm for AWS, “Being able to implement everything in record time and forget about performance issues is a blessing. We love the ease of use and scalability of AWS."

To learn more, visit http://www.esa.int/ .

Added April 5, 2011