AWS Public Sector Blog

46 new datasets available on the Registry of Open Data on AWS

https://app.asana.com/1/8442528107068/project/1207199896111772/task/1218756094593739?focus=true

The Amazon Web Services (AWS) Open Data Sponsorship Program makes high-value, cloud-optimized datasets publicly available on AWS. AWS works with data providers to democratize access to data by making it available to the public.

People can use it for analyzing data on AWS or developing new cloud-based techniques, formats, and tools that lower the cost of working with data. This is to encourage the development of communities that benefit from access to shared datasets.

Through the AWS Open Data Sponsorship Program, customers are making over 400 PB of high-value, cloud-optimized data available for public use.

These publicly available datasets can be found in the Registry of Open Data on AWS and are also discoverable on AWS Data Exchange. This quarter, AWS released 46 new or updated datasets.

What are people doing with the Registry of Open Data on AWS?

Organizations are using the Registry of Open Data on AWS in many different ways, including:

  • Common Crawl and AWS Open Data built the foundation for the AI revolution. The AWS Open Data Sponsorship Program has hosted the Common Crawl open repository of web data since January 2012. It has become one of the most important sources of training data for the large language models (LLMs) reshaping industries. A 2024 Mozilla Foundation study found that 64 percent of the 47 major LLMs published between 2019 and 2023 used filtered versions of Common Crawl data for training. The dataset has been cited in over 13,000 research papers.
  • dClimate announces that TESSERA, a global geospatial AI foundation model, is being made openly available through the Amazon Sustainability Data Initiative (ASDI) and AWS Open Data Sponsorship Program. With compute and hosting support from AWS, dClimate is completing TESSERA’s global historical coverage and publishing the resulting analysis-ready data on Amazon Simple Storage Service (Amazon S3) for public use, free of charge.
  • Digital Earth Africa has partnered with AWS through the Open Data on AWS and Amazon Sustainability Data Initiative programs to host and process satellite data on cloud infrastructure in Cape Town, making Earth observation usable at continental scale. Over 6 years, it has grown to 11 PB of open satellite data delivering 12 continental services to more than 38,000 users and over 50 government ministries. It has built successive layers of water and coastline intelligence—Water Observations from Space, the Waterbodies Monitoring Service, the new Water Quality Monitoring Service, and the Coastlines Monitoring Service, each enabling real-world decisions.
  • ZipCheckup announced that the US Home Environmental Risk Compilation is now openly accessible through AWS Open Data. The dataset covers machine-parsed Consumer Confidence Reports for more than 9,700 water systems across 2023 and 2024, including PFAS contaminant tables; lead service line records aggregated from 94 state and utility inventories, totaling 18.4 million address-level records; a ZIP-level summary layer covering more than 41,000 ZIP codes across 17 environmental risk categories; and companion layers built from EPA SDWIS, UCMR5, FEMA NRI, USGS, CDC, and Census sources.

Artemis and Image Safari agricultural imaging datasets from Alliance of Bioversity International and CIAT

The Alliance of Bioversity International and CIAT is a global research-for-development organization focused on sustainable agriculture, climate resilience, and agricultural biodiversity. Artemis and Image Safari are a unified agricultural image collection comprising more than 7.4 million fieldimages across two complementary datasets within the Registry of Open Data. The collection spans 18 crop species and 11 countries across sub-Saharan Africa, Latin America, and the US, with emphasis on smallholder and structured phenotyping farming systems that remain underrepresented in public imaging.

Together, the datasets combine crop imagery, structured metadata, and annotations to support agricultural foundation models, plant disease detection, growth stage recognition, crop identification, agricultural monitoring, and cross-environment generalization research.

Artemis and Image Safari join 45 other new or updated datasets on the Registry of Open Data on AWS in the following categories:

Climate and weather

Geospatial

Life sciences

AI/ML

How can you make your data available?

The AWS Open Data Sponsorship Program covers the cost of storage for publicly available high-value, cloud-optimized datasets. We work with data providers who seek to:

  • Democratize access to data by making it available for analysis on AWS
  • Develop new cloud-based techniques, formats, and tools that lower the cost of working with data
  • Encourage the development of communities that benefit from access to shared datasets

Learn how to propose your dataset to the AWS Open Data Sponsorship Program.

Learn more about open data on AWS.