Tag: open data

Since 1971, Canada Centre for Mapping and Earth Observation (CCMEO) at Natural Resources Canada (NRCan) has accumulated an Earth observation (EO) data archive in excess of two petabytes (PB). NRCan wanted to modernize its geospatial offerings at a faster pace, so they turned to the AWS Snow Family on AWS to migrate their large volume of data.

The theme of this year’s Earth Day is “Invest in our planet.” In celebration of this year’s theme, we want to highlight the important sustainability work featured in the AWS documentary series, Climate Next, which explores the ways organizations and communities in four distinct regions are investing in cloud-powered solutions to fight climate change.

The Materials Project at Lawrence Berkeley National Laboratory (LBNL) is an open database that offers information about material properties, or, all the elements and substances that make up the products we use every day. By harnessing the power of the Department of Energy’s (DOE) high-performance scientific computing and state of the art electronic structure methods, the Materials Project provides open web-based access on AWS to computational datasets on both known and potential materials, along with powerful analysis tools to help discover, inspire, and design new materials.

The AWS Open Data Sponsorship Program makes high-value, cloud-optimized datasets publicly available on Amazon Web Services (AWS). Our full list of publicly available datasets are on the Registry of Open Data on AWS. This quarter, we released 13 new or updated datasets including CMIP5, 1950s US Decennial Census, and open genomics data for Galaxy. Read on for some highlights.

As part of the Amazon Sustainability Data Initiative (ASDI), AWS invited Rafael Monge Vargas, director of the National Center of GeoEnvironmental Information (CENIGA) at the Costa Rica’s Ministry of Environment and Energy (MINAE), to share how his team is helping advance conservation and economic development in Costa Rica and how they utilize ASDI and AWS to support these efforts.

On April 1, the US National Archives and Records Administration (NARA) released the 1950 Census data to the general public. Census data is released 72 years after a census is conducted, and it has been 10 years since the last census data for the 1940 Census was publicly released. With the support of cloud technologies, this release marks a number of important firsts. AWS is honored to support the release of the 1950 Census and help make this data available to the public.

As part of the Amazon Sustainability Data Initiative (ASDI), AWS invited Nils Helset, co-founder and chief executive officer (CEO) of DigiFarm, to share how AWS Cloud technology and open data support DigiFarm’s efforts in precision farming to make agricultural practices more sustainable and efficient.

How do we avoid the next global pandemic? For researchers collaborating with the University of British Columbia Cloud Innovation Center (UBC CIC), the answer to that question lies in a massive library of genetic sequencing data. But there is a problem: the data library is so massive that traditional computing can’t comprehensively analyze or process it. So the UBC CIC team collaborated with computational virologists to create Serratus, an open-science viral discovery platform to transform the field of genomics—built on the massive computational power of the Amazon Web Services (AWS) Cloud.

The AWS Open Data Sponsorship Program makes high-value, cloud-optimized datasets publicly available on AWS. We work with data providers to democratize access to data by making it available to the public for analysis on AWS; to develop new cloud-native techniques, formats, and tools that lower the cost of working with data; and to encourage the development of communities that benefit from access to shared datasets. This quarter, we released 19 new or updated datasets like validated OpenStreetMap data, bioacoustic data, COVID-19 detection data, and more.

Terabytes of Earth Observation (EO) data are collected each day, quickly leading to petabyte-scale datasets. By bringing these datasets to the cloud, users can use the compute and analytics resources of the cloud to reliably scale with growing needs. In this post, we show you how to set up a Pangeo solution with Kubernetes, Dask, and Jupyter notebooks step-by-step on Amazon Web Services (AWS), to automatically scale cloud compute resources and parallelize workloads across multiple Dask worker nodes.