WillyWeather Runs Nowcasting Model for Weather Predictions on AWS
Location-Specific Predictions for Short-Term Time Frames
Whether for leisure pursuits or outdoor business, reliable rain alerts can greatly impact people’s behavior and businesses’ ability to adapt to inclement weather. Weather forecasting is a highly variable practice though, subject to capricious forces that can change at a moment’s notice. Traditional weather forecasts help people make plans for the weekend, but the models don’t typically paint an accurate picture of short-term weather changes. Nowcasting enhances short-term weather forecasting to produce location-specific predictions for the next 0 to 6 hours.
WillyWeather is one of Australia’s largest weather prediction providers, and also serves users in the UK and US. It developed a nowcasting model in 2020 that uses real-time data from 70 radars across Australia to predict rain and small-scale storms. The company has been using Amazon Web Services (AWS) since 2018 and reached out to AWS to help develop its nowcasting alert system.
Our whole database and nowcasting system require no manual input, which is pretty amazing.”
Technical Director and Founder, WillyWeather
Dedicated Developer Partner with Machine Learning Expertise
WillyWeather has a highly skilled team of developers and engineers, but this was its first project using machine learning (ML). By experimenting with various approaches, it’d developed two separate nowcasting models over the past two years, but neither achieved the level of accuracy it wanted. To help WillyWeather improve its accuracy, AWS introduced Intellify, an AWS Advanced Consulting Partner with artificial intelligence (AI) and ML expertise.
“What we were missing in the past was a dedicated team of developers who have years of experience building complex AI and ML systems, and we found that in Intellify,” says David Allen, technical director and founder of WillyWeather. “The Intellify team has been transparent and accurate about timelines and, along with AWS, they immediately shared our enthusiasm about this project.”
The first phase of development involved extensive consultation and testing to find the right technology fit. The initial approach trialed deep learning models, which yield a high level of accuracy but would have required a higher level of investment in terms of time and resources to deploy. However, WillyWeather wanted to keep project and maintenance costs low because it planned to make nowcasting results free for its users. Intellify’s alternative solution was to run AWS serverless technology with respect to this particular use case and project priorities.
Real-Time Rain Notifications for 5 Million Users
The nowcasting models use AWS Lambda serverless code linked to real-time radar imagery data. When rain is detected, AWS Lambda triggers the nowcasting model, which generates weather predictions every 2 minutes and then records the actual weather for comparison. WillyWeather uses Amazon Simple Storage Service (Amazon S3) for low-cost storage of actual and predictive data.
WillyWeather’s 5 million active monthly users receive weather alerts on their mobile devices via Amazon Simple Notification Service (Amazon SNS). The company uses Amazon Simple Queue Service (Amazon SQS) to manage notification queues at scale, which ensures there’s no delay in relaying alerts.
Its engineering team also set up notifications in Amazon CloudWatch to receive alerts of any abnormal activity, such as failed notifications. “Having direct access to AWS principal solutions architects helped us find the right service and product for each use case. This, in combination with Intellify’s experience in building AI- and ML-based systems, has contributed to the success of our nowcasting project,” Allen says.
Serverless Models Support Cost-Optimized Scaling
Serverless infrastructure was an ideal fit for AI-based nowcasting. On an average day, rain is only detected at a few radar stations, which doesn’t require the nowcasting models to run at full speed on dedicated servers. However, a vast amount of on-demand processing power is needed to run the algorithms once rain is detected, and traffic to the WillyWeather app and website can spike 300 percent during severe weather warnings. Rapid infrastructure scaling during these times is critical. With AWS Lambda, WillyWeather can scale the nowcasting model infinitely in response to event triggers. It pays only for compute time consumed, avoiding costly over-provisioned infrastructure.
WillyWeather also adjusted its database architecture to suit its nowcasting workloads, and it is now using Amazon Aurora Serverless with Amazon Relational Database Service (Amazon RDS) for this purpose. Load balancing is automatic, and its database integrates well with serverless code execution on AWS Lambda. “Our whole database and nowcasting system require no manual input, which is pretty amazing,” Allen says. “The system responds in real time and is highly available. It also provides the reliability and scalability to ensure all our 5 million users will be notified about any significant weather events as soon as possible.”
Visual Timelines Show Rain Patterns and Intensity
WillyWeather is now finalizing the second stage of its nowcasting project—a notification engine to alert users of incoming rain. Users can tap on push notifications to visualize rain moving over a map of the country or zoom in to view the area where they’re currently located. Users will also see a graph with an adjustable timeline predicting the intensity of incoming rain.
“This system marries well with our mobile platform,” Allen comments. “The nowcasting model adjusts to users’ coordinates and notifies them about changes in the weather system relevant to their position.” Mobile users currently account for roughly half of WillyWeather’s customer base, but the segment is rapidly growing.
Reflecting on the success of the project to date, Allen credits both Intellify and AWS for their support in bringing the nowcasting system to life in a short time frame. “In the future, we’ll probably be faster to recognize any internal limitations and more inclined to reach out to AWS and its partners to develop specific projects.”
To learn more, visit Serverless on AWS
WillyWeather is an Australian weather prediction company also operating in the UK and US. Its website, iOS, and Android platforms support more than 5 million monthly users.
Benefits of AWS
- Scales to run AI models for 5 million monthly users
- Optimizes costs with serverless on-demand infrastructure
- Develops a nowcasting system quickly with expert support
- Manages notification queuing to send weather warnings in parallel to all users
- Supports dynamic location data and expanding mobile customer base
AWS Services Used
AWS Lambda is a serverless compute service that lets you run code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes.
Amazon Aurora Serverless
Amazon Aurora Serverless is an on-demand, auto-scaling configuration for Amazon Aurora. It automatically starts up, shuts down, and scales capacity up or down based on your application's needs. It enables you to run your database in the cloud without managing any database capacity.
Amazon Simple Notification Service
Amazon Simple Notification Service (Amazon SNS) is a fully managed messaging service for both application-to-application (A2A) and application-to-person (A2P) communication.
Amazon Simple Queue Service
Amazon Simple Queue Service (SQS) is a fully managed message queuing service that enables you to decouple and scale microservices, distributed systems, and serverless applications.
Companies of all sizes across all industries are transforming their businesses every day using AWS. Contact our experts and start your own AWS Cloud journey today.