weathernews

Weathernews makes weather forecasts in 10-minute intervals with over 90% accuracy on AWS HPC clusters

kr_quotemark

Using AWS, we’re able to forecast rainclouds up to 30 hours ahead in 10-minute intervals. The AWS platform allows engineers to experiment with their own ideas and create new services."

Tomohiro Ishibashi
Senior Managing Executive Officer, Weathernews Inc.

Weathernews
Weathernews is the world's largest private weather service provider and the global leader in weather forecasting. Using AWS ParallelCluster to manage its computing resources, the company produces unprecedented high-resolution forecasts, boasting 10-minute forecast intervals up to 30 hours ahead. Scalable high-performance computing (HPC) environments allow the company to procure large amounts of computing resources and adapt to load fluctuations flexibly. With Amazon Web Services (AWS), Weathernews achieves over 90 percent forecast accuracy.

Aiming for weather forecasts with unprecedented detail

Founded in 1986, Weathernews is the world's largest private meteorological company, servicing around 50 countries. The sailing, aviation, railway, roads, and logistics industries use its services, and the Weathernews smartphone app is also well adopted.
 
The meteorological industry typically obtains its basic information from government agencies. However, over its history, Weathernews has built its own infrastructure for observation, communication networks, image processing, and delivery. To achieve accurate weather forecasting, the company developed a unique weather prediction model which leveraged artificial intelligence (AI) technology to analyze weather reports from radars and users, for forecasts in 10-minute intervals up to three hours ahead. Weathernews was initially providing 15-hour forecasts in one-hour intervals, but at one-hour intervals, rain cloud graphics became blurry and difficult to comprehend. And with sudden torrential rains and other weather changes becoming more common in recent years, the need for higher resolution long-term forecasts increased. “As meteorologists, we dream of longer-term forecasts with greater detail,” says Tomohiro Ishibashi, Weathernews Senior Managing Executive Officer.
 
To achieve this vision, Weathernews began working on higher-resolution forecasts with 10-minute intervals, increasing the forecast range to 30 hours, also viewable in 10-minute intervals, and improving forecast frequency from 6 hours to 3.

Provisioning large compute resources flexibly

Weathernews’ main challenge was to procure the large amount of computing resources needed for forecasts.

“The conventional method of adding on-premises resources would require a massive investment, and the extra servers could increase the potential for failures and add to operating workloads,” explains Kohei Sakamoto, Leader of the Weathernews Forecast Center Development Team. In addition, typhoons and sudden downpours are more common in Japan from June to October, requiring more computational resources than other seasons. It was difficult for the company to flexibly respond to these load fluctuations with its on-premises environment.

In 2018, Weathernews began examining next-generation original weather models using AWS ParallelCluster. After thorough validation, the company decided to adopt the solution in April 2020.

“We used actual model calculations to validate the relationship between the number of AWS ParallelCluster instances and performance, the effectiveness of Elastic Fabric Adapter (EFA), and the impact of instance type on processing speed,” says Kazunari Takahashi, Section Leader of the Weathernews Forecast Center Development Team. “Because the system required high performance and stable operation with a reasonable cost, we discussed configurations that could meet availability, cost, and performance requirements.” Ultimately, Weathernews chose a configuration leveraging multiple AWS regions and Amazon EC2 spot instances.

Providing over 90% forecast accuracy, higher-resolution forecasts, and enhanced alert functions

With the additional compute resources on AWS ParallelCluster, Weathernews achieved its initial goal of forecasting every 10 minutes up to 15 hours ahead, and launched the Weathernews smartphone app for premium subscribers in July 2020. Forecasts subsequently improved to up to 30 hours ahead in 2022. And the improvements aren’t limited to time: spatial resolution has been refined from the previous one-to-five km mesh to a 250-meter mesh. These higher resolutions leverage new AI technologies which were trained on AWS.

The ability to forecast 30 hours ahead at 10-minute intervals and the added intuitiveness was met with a great response, generating an increase in the number of paid users. “For example, you can clearly see the rotation of a typhoon. By building upon past weather patterns, we can see how the weather will change. No one has been able to see this information before, so it’s creating a considerable impact,” says Ishibashi.

Further improvements include a rain cloud alert function which enables users to respond flexibly to sudden weather changes with the latest forecast for their current location.

The new system has also boosted forecast reliability, with accuracy rates of over 90% (daily precipitation rate) in 2020 and 2021. The ability to dynamically procure required compute resources has also sped up improvements to the predictive model.

Growing innovation from internal experiments

The benefits of AWS are more than just longer-term, more granular weather forecasts: “From a business perspective, we believe that having our meteorologists and engineers experiment with their own ideas will enhance forecasting that wouldn’t have been possible otherwise,” explains Ishibashi. “This is because innovative ideas that shape the future start from play. If this requires heavy investment, it's difficult to take the first step. But AWS lets us lower the risk and minimize cost if things don’t work out.” 

According to Ishibashi, AWS also makes it easier to collaborate externally and incorporate new technologies. “We can look forward to innovations from new challenges with AWS supporting companies like Weathernews.”

Weathernews

Behind the Innovation. AWS is How.

Watch the Ad