HSAT Cuts Satellite Data Processing Times by 80% Using AWS

Heimdal Satellite Technologies (HSAT) analyzes satellite and geospatial data to provide clients with valuable insights and predictions about crops and logistics as well as environmental, social, and corporate governance (ESG) on a national and global scale. The company cut compute costs by 70 percent and data processing time by 80 percent by moving its data processes to AWS. This supports its mission to process information at global scale at a fraction of the cost of its previous approach.

Problem solving is the key to a successful business. Customers come to a business looking for solutions to their problems, both large and small. Heimdal Satellite Technologies (HSAT) has built its business on understanding and solving problems on a global scale.

The space age has given us access to huge amounts of data from satellites and space missions—data that can drive better business decisions and save money, as well as save lives and protect the environment. But data needs to be processed and analyzed to provide the insight that organizations need. That’s why organizations, from traders and farmers, to insurers and NGOs, turn to HSAT.

The company has operations in the UK and Norway. It analyzes petabytes of satellite imagery and geographical and geospatial data to provide accurate information about what is being shown and generate insights about land use. This allows customers to make better-informed decisions.

Like its namesake, the Norse god Heimdal, renowned for foresight and keen vision, HSAT looks for connections and insights that others overlook.

HSAT uses a powerful and complex data-processing algorithm to ingest petabytes of data, mapping entire continents and analyzing every 10m x 10m area of land. This enables HSAT to gain an understanding of how land is being used in precise detail, on a global scale. For example, measuring what crops are being grown, their quality, and yield, across an entire continent. This requires tens of trillions of calculations.

Customers from a range of different industries—including trading, agriculture, and food and drink manufacturers—require insights into the availability, pricing, and sustainability of crops and food around the word. HSAT provides these insights via dashboards, alerts, or custom APIs, to ensure clients have the right information at the right time.

But HSAT had a problem of its own to solve. Processing and analyzing all that data was taking too long and costing too much, sometimes as much as $5,000 per day. For HSAT to provide the insight its customers expect, it needed fast and cost effective computer processing resources at scale, which is why it turned to Amazon Web Services (AWS) for help.

The information generated by HSAT gives customers insights into things that can affect their business. For example, it tracks sugar production and yield worldwide, and provides this information to traders, sugar producers, food and drink producers, and supermarkets. Insurance companies that insure crops or traders that invest in sugar futures, meanwhile, use this information to make better decisions.

HSAT can also provide near real-time analysis of events, such as the risk of forest fires or hurricanes, on a region’s crops.

“We’re about predicting events on a global scale. We speak to clients, find out their key issues, and then get the data to solve that problem. We are entirely driven by a ‘client first’ approach,” explains HSAT co-founder Rob Weston.

The company previously worked with data from various sources, including the European Space Agency (ESA), NASA, drones, and weather data, as well as information sourced directly from teams on the ground. This data was then moved to its existing cloud provider to run analysis on single, large-scale virtual machines.

However, this process was time-consuming and required a huge spend on compute resources to achieve the necessary scale. To tackle these challenges HSAT decided to investigate the potential cost and time savings of running its processes on AWS instead.

After running a proof of concept between December 2020 and January 2021, HSAT found that using AWS Batch enabled analysis to be done much more quickly and more cost effectively thanks to the ability to run many analyses at the same time, and to make effective use of Amazon EC2 Spot Instances.

“With AWS we can get things running really quickly and easily. We can work at massive scale,” says Weston.

For example, HSAT can spin up 6,000 CPUs through AWS Batch to process all ESA satellite data on Brazil—equivalent to 30-40 trillion calculations—and get the results on the same day.

By using AWS, HSAT can provide large-scale data analysis at a fraction of the cost of its previous approach. AWS Batch has helped cut processing time for analyzing images of large areas of land by as much as 80 percent, while cutting costs of compute hours by as much as 70 percent.

Another reason why HSAT decided to work with AWS was the company’s willingness to help. “The number one reason we moved is the support we got from AWS,” says Weston. “The depth of technical resourcing, from code, to technical support, to the ‘big picture’ view is just phenomenal. Overall, it’s the integration, combined with the volume of data we can access, the scalability, and the support we get.”

The improved speed and scalability that AWS makes possible means HSAT can take on new problems that customers want to solve. This includes seeking insights to address environmental concerns, which have become a major issues for businesses, governments, investors, and the public.

For example, the company tracks deforestation and the development of forest fires in real time, providing environmental and government information to help tackle the blazes.

The major consumer brands that work with HSAT can use some of this information to identify unethical or criminal efforts to clear land by forest fires, and ensure they make ethical and sustainable decisions about where they buy their raw materials.

The breadth and depth of insight that HSAT can deliver to clients is growing all the time. But none of this would be possible without AWS, according to Weston.

“We would not be a viable business without AWS. It’s as simple as that,” he says.