Equans Constructs Smart Building Data Analytics System on AWS with Necko Technologies

Equans BeLux, expert in the energy and services sector, worked with AWS Partner Necko Technologies to build a flexible system on Amazon Web Services (AWS) that can ingest and serve 1.5 TB of smart buildings monitoring data for 250 clients. The system must ingest and serve more than 10 different data formats to a variety of customer monitoring systems in near-real-time. Although data loads can be predicted, the system must be agile enough to scale up quickly for new clients or to process additional sets of data. It must also be able to rapidly alter APIs and the format that data is converted into if customers change to a new management tool.

Equans BeLux is made up of several businesses, including a Smart Buildings division mainly working with commercial and office management companies. This division wanted a better way to ingest and serve data to its customers. They needed more flexibility to make full use of the smart building data it collected.

The system needs to ingest 10 streams of data from different monitoring devices in a variety of formats. These streams must then be processed and served back to customers using a range of different third-party monitoring and management software and dashboards. With 250 clients and data logs that transmit every 15 minutes, this adds up to a large amount of data. The project also migrated stored data, some dating back to 2014.

Having worked with AWS Partner Necko Technologies on an earlier project, Equans asked the company to help it build a new system on AWS. “Necko are really efficient and knowledgeable, I’ve learnt a lot from them during this project,” says Julian Fickers, project engineer at Equans. “Now we’ve got a very maintainable infrastructure that just works.”

The system must deal with thousands of pieces of data every few minutes and it must work rapidly to provide customers with the alerts and insights they need within a few seconds. That speed is provided by using Amazon DynamoDB, a fast, flexible NoSQL database service for single-digit millisecond performance at any scale.

Necko Technologies used Amazon API Gateway, a fully managed service to create, publish and maintain APIs at any scale, and AWS Transfer Family, to easily manage and share data with simple, secure, and scalable file transfers. The data must be converted to standard formatting and compressed and then transferred to Amazon Simple Storage Service (Amazon S3), object storage built to retrieve any amount of data from anywhere.

The entire system is serverless and uses AWS Lambda, which lets Equans run code without thinking about servers or clusters. In addition, the system uses infrastructure as code (IaC) so it can easily be deployed and managed. Data is converted and compressed before being forwarded onto customers.

Credit: Equans

The new system has changed how Equans can work with its customers. “This is a more developed version compared to what we had before,” says Fickers. “We can link to just about any systems a customer might have. We can just say ‘yes’ to them now. We have actually started a new smart buildings solution team now because we have the flexibility and ability to actively help our customers.”

The event-driven system uses AWS Step Functions, which offers visual workflows for distributed applications, to provide serverless orchestration and design workflows. That means the company can scale quickly when required without incurring heavy overheads. This scalability allows Equans to easily onboard new customers and new buildings, or make data available to different systems. It automatically buffers data so it doesn’t overload customer systems.

Equans does not currently archive data but knows it could link to cheaper AWS archive and ‘cold storage’ services in the future.

A new system feature is an API that will allow customers to access and query data directly. “The next step is to start using some AWS analytics services,” says Jérôme Dauge, co-founder of Necko Technologies. “We could provide tailor-made reports because the data can now be queried direct in AWS.”

Equans is in a strong position to innovate with its service running on AWS. Its systems can easily link to other AWS services to provide improved customer services, more intelligent use of sensor data, and more analytical services in the future.

In addition to the freedom to innovate, Equans values the minimal management overhead of the new system. “When AWS is configured properly, it never fails,” says Fickers. “It’s not like a server that needs rebooting or software that crashes, we can just relax and sleep well at night.”