WillyWeather Runs Nowcasting Model for Weather Predictions on AWS

Whether for leisure pursuits or outdoor business, reliable rain alerts can greatly impact people’s behavior and businesses’ ability to adapt to inclement weather. Weather forecasting is a highly variable practice though, subject to capricious forces that can change at a moment’s notice. Traditional weather forecasts help people make plans for the weekend, but the models don’t typically paint an accurate picture of short-term weather changes. Nowcasting enhances short-term weather forecasting to produce location-specific predictions for the next 0 to 6 hours.

WillyWeather is one of Australia’s largest weather prediction providers, and also serves users in the UK and US. It developed a nowcasting model in 2020 that uses real-time data from 70 radars across Australia to predict rain and small-scale storms. The company has been using Amazon Web Services (AWS) since 2018 and reached out to AWS to help develop its nowcasting alert system.

WillyWeather has a highly skilled team of developers and engineers, but this was its first project using machine learning (ML). By experimenting with various approaches, it’d developed two separate nowcasting models over the past two years, but neither achieved the level of accuracy it wanted. To help WillyWeather improve its accuracy, AWS introduced Intellify, an AWS Advanced Consulting Partner with artificial intelligence (AI) and ML expertise.

“What we were missing in the past was a dedicated team of developers who have years of experience building complex AI and ML systems, and we found that in Intellify,” says David Allen, technical director and founder of WillyWeather. “The Intellify team has been transparent and accurate about timelines and, along with AWS, they immediately shared our enthusiasm about this project.”

The first phase of development involved extensive consultation and testing to find the right technology fit. The initial approach trialed deep learning models, which yield a high level of accuracy but would have required a higher level of investment in terms of time and resources to deploy. However, WillyWeather wanted to keep project and maintenance costs low because it planned to make nowcasting results free for its users. Intellify’s alternative solution was to run AWS serverless technology with respect to this particular use case and project priorities.

The nowcasting models use AWS Lambda serverless code linked to real-time radar imagery data. When rain is detected, AWS Lambda triggers the nowcasting model, which generates weather predictions every 2 minutes and then records the actual weather for comparison. WillyWeather uses Amazon Simple Storage Service (Amazon S3) for low-cost storage of actual and predictive data.

WillyWeather’s 5 million active monthly users receive weather alerts on their mobile devices via Amazon Simple Notification Service (Amazon SNS). The company uses Amazon Simple Queue Service (Amazon SQS) to manage notification queues at scale, which ensures there’s no delay in relaying alerts.

Its engineering team also set up notifications in Amazon CloudWatch to receive alerts of any abnormal activity, such as failed notifications. “Having direct access to AWS principal solutions architects helped us find the right service and product for each use case. This, in combination with Intellify’s experience in building AI- and ML-based systems, has contributed to the success of our nowcasting project,” Allen says.

Serverless infrastructure was an ideal fit for AI-based nowcasting. On an average day, rain is only detected at a few radar stations, which doesn’t require the nowcasting models to run at full speed on dedicated servers. However, a vast amount of on-demand processing power is needed to run the algorithms once rain is detected, and traffic to the WillyWeather app and website can spike 300 percent during severe weather warnings. Rapid infrastructure scaling during these times is critical. With AWS Lambda, WillyWeather can scale the nowcasting model infinitely in response to event triggers. It pays only for compute time consumed, avoiding costly over-provisioned infrastructure.

WillyWeather also adjusted its database architecture to suit its nowcasting workloads, and it is now using Amazon Aurora Serverless with Amazon Relational Database Service (Amazon RDS) for this purpose. Load balancing is automatic, and its database integrates well with serverless code execution on AWS Lambda. “Our whole database and nowcasting system require no manual input, which is pretty amazing,” Allen says. “The system responds in real time and is highly available. It also provides the reliability and scalability to ensure all our 5 million users will be notified about any significant weather events as soon as possible.”

WillyWeather is now finalizing the second stage of its nowcasting project—a notification engine to alert users of incoming rain. Users can tap on push notifications to visualize rain moving over a map of the country or zoom in to view the area where they’re currently located. Users will also see a graph with an adjustable timeline predicting the intensity of incoming rain.

“This system marries well with our mobile platform,” Allen comments. “The nowcasting model adjusts to users’ coordinates and notifies them about changes in the weather system relevant to their position.” Mobile users currently account for roughly half of WillyWeather’s customer base, but the segment is rapidly growing.

Reflecting on the success of the project to date, Allen credits both Intellify and AWS for their support in bringing the nowcasting system to life in a short time frame. “In the future, we’ll probably be faster to recognize any internal limitations and more inclined to reach out to AWS and its partners to develop specific projects.”

To learn more, visit Serverless on AWS