Building AI-powered weather forecasting tools with Open Data on AWS

Figure 1. Hurricane Melissa captured by NOAA Geostationary Operational Environmental Satellites (GOES) 19 on October 26, 2025

From devastating hurricanes to prolonged droughts, the frequency and intensity of extreme weather events around the world are rising dramatically—presenting unprecedented challenges to communities, businesses, and governments worldwide. These dramatic weather phenomena are reshaping how we think about climate resilience and adaptation. Accurate and timely weather forecasting has become more than a convenience—it’s now a critical tool for protecting lives, infrastructure, and economic stability. According to the Hydromet Gap Report 2021, “an estimated 23,000 lives per year could be saved and potential annual benefits of at least US $162 billion could be realized by improving weather forecasts, early warning systems, and climate information.”

Although traditional weather forecasting methods have served us well, they often require substantial computational resources and time to deliver results. In this post, we explore how Brightband is revolutionizing this field by combining AI with Open Data on AWS to create faster, more accessible, and highly accurate weather forecasting solutions. When data is shared on Amazon Web Services (AWS), anyone can analyze it and build services on top of it. Sharing data in the cloud lets data users such as Brightband spend more time on data analysis rather than data acquisition.

Brightband: Democratizing advanced ML weather forecasting

Brightband is at the forefront of a new era in weather forecasting, developing accessible AI-powered tools designed to help humanity adapt to increasingly extreme weather conditions. The startup recently won the 2024 Compute for Climate Fellowship, a global funding initiative created by UNESCO’s AI research center (IRCAI) and AWS. This fellowship supports promising climate tech startups that use cloud computing and AI to tackle climate change. Through this support, Brightband is building a comprehensive, end-to-end AI-based Earth System specifically for probabilistic forecasting.

What distinguishes Brightband’s approach is their pioneering work in weather analysis and forecasting. Unlike most services that rely exclusively on analyses from National Oceanic and Atmospheric Administration (NOAA) or European Centre for Medium-Range Weather Forecasts (ECMWF) as starting points, Brightband has developed the capability to generate their own weather analyses directly from raw observations. Every 6 hours, they typically run a comprehensive suite of forecasts using three different initial conditions: their own analysis alongside those from NOAA and ECMWF. This unique approach enables direct comparison of forecast outcomes from different starting points. Their machine learning (ML) models maintain impressive efficiency, producing forecasts within minutes of receiving initial conditions, which is substantially faster than traditional methods that often take hours to complete. When evaluated using standard meteorological assessment techniques, these ML models have proven to be not only competitive with but frequently more skillful than top models from NOAA and ECMWF.

Figure 2: ML approach with diffusion model: Generate a realistic trajectory of the atmosphere that is also consistent with observations. Photo credit: Brightband

This achievement in accuracy doesn’t come at the cost of computational efficiency—quite the opposite. Although traditional weather modeling approaches require access to hundreds or thousands of computing cores on high-performance systems, Brightband’s ML models can run on a single enterprise-grade GPU, completing entire forecast rollouts in mere minutes. This translates to operational costs of less than a dollar per forecast, making advanced weather forecasting accessible to a broader range of users, from academic institutions and government organizations to private companies of all sizes. This democratization of weather forecasting technology could have far-reaching implications for climate adaptation and resilience planning worldwide. The ability to run forecasts on smaller hardware also means that organizations can produce forecasts that are specific to their location or industry and can process data directly from their own sources—such as local weather stations, satellites, or weather balloons—without relying on external computing resources.

The benefits extend beyond economics. By dramatically reducing energy and computing requirements compared to traditional supercomputing methods, this ML-driven approach offers a more environmentally sustainable path for weather forecasting.

NOAA data on AWS

Brightband uses data from NOAA’s Global Forecast System (GFS) dataset, available through the Registry of Open Data on AWS as part of their operational forecast suite. Previously, they needed to rely on NOAA’s Operational Model Archive and Distribution System (NOMADS), which presented numerous challenges including downtimes and limited historical data availability through its 10-day rolling archive system. Open Data on AWS has transformed this landscape entirely, providing them with a seamless way to access this key data. They can also use the archive of data available as part of their comprehensive evaluation of forecast model performance.

Brightband has developed sophisticated data processing pipelines on AWS that are immediately triggered when new NOAA data becomes available on Open Data on AWS. These pipelines handle a key aspect of weather data processing: converting raw GFS data from its original GRIB format to the more efficient Zarr format. This conversion is crucial because it optimizes the data for distributed computing environments and enables more efficient data access patterns. The system processes approximately 5 years of historical data, allowing for comprehensive model evaluation and refinement.

The benefits of this AWS powered architecture are substantial. Direct access to public Amazon Simple Storage Service (Amazon S3) buckets eliminates the need for data transfer and reduces both time and costs. The reliability and scalability of AWS services mean that Brightband can consistently generate multiple forecasts daily across their half-dozen ML-based weather prediction models (MLWP), running every 6 hours.

Start building now, with Open Data on AWS

To accelerate innovation in weather and climate analytics, we’ve created a comprehensive QuickStart Jupyter notebook that demonstrates how to access and analyze NOAA GFS data on the Registry of Open Data on AWS. This notebook serves as a practical guide for developers, researchers, and data scientists who want to begin working with weather data, lowering the barrier to entry for weather and climate analytics.

The notebook provides detailed guidance on several key areas:

• Establishing connection and accessing NOAA GFS data from Registry of Open Data on AWS
• Understanding and working with different weather data formats
• Implementing basic weather data analysis techniques
• Creating meaningful visualizations of weather patterns and trends
• Implementing best practices for efficient data processing and analysis

Through this notebook, you can learn how to use the same powerful datasets that organizations such as Brightband use to build their forecasting systems.

Ready to start building? Launch the QuickStart notebook in Amazon SageMaker AI Studio today or explore it in SageMaker Studio Lab if you don’t have an AWS account. To create an account, visit Sign up for AWS.