Hitachi rebuilt the foundation for material development solutions provided by the cloud with a serverless architecture

Hitachi's material development solution applies materials informatics (MI) to derive material development guidelines by leveraging machine learning, AI, and other technologies using diverse material and experimental data. In recent years, competition for the development of new materials has intensified, necessitating rapid and cost-effective development processes. However, in the past, the common method involved iterative experiments based on expert experience and theory, leading to increased development time and costs, especially when aiming to develop highly functional materials. To address this issue, Hitachi released a materials development solution using MI in 2017.

“Researchers engaged in material development have developed it as a general-purpose service that can be used on a daily basis without advanced MI expertise,” says Mr. Masayuki Fukuoka from the Digital Solution Promotion Department, Public Infrastructure Solutions Division.

The material development solutions consist of four types: an ‘analysis environment provision service’ that offers a cloud-based environment for material data analysis; an ‘analysis support service’ that conducts analyses on behalf of customers using their material data; a ‘DX support service’ that defines a platform to facilitate digital transformation; and an ‘experimental data collection service’ that records and manages experimental data.

Among them, the “Analysis Environment Provision Service” is a service that analyzes and visualizes material data, such as simulation data and experimental data held by customers, in a cloud environment provided by Hitachi. Customers can construct prediction models in an analysis environment while operating Hitachi's proprietary AI tools on a web browser, predict results for unknown data using the prediction models, and estimate variables to achieve target values from the prediction models.

AWS has been utilized as the platform for the analytics environment since the inception of service delivery in 2017. “Previously, we built an analysis platform in an on-premise environment, but when making it a service, we built it using AWS, which has know-how and development experience in our dedicated cloud team, for the purpose of shortening the lead time for server procurement and reducing operational load. With AWS, there is a template Baseline Environment on the AWS (BLEA) that provides a secure baseline, and customers can use it with peace of mind,” says Mr. Teruaki Obata of the Digital Solution Promotion Department, Public Infrastructure Solutions Division.

However, since the initial environment was a virtual server-based configuration using Amazon Elastic Compute Cloud (Amazon EC2), cost and speeding up service delivery have emerged as new issues. Therefore, the Digital Solution Promotion Department decided to rebuild the architecture.

"Until now, the analysis environment consisted of a front end for the screen display area and a back end for the analysis execution area, providing an independent environment for each customer." However, since there is no need to keep the server running all the time in the analysis execution area, we decided to separate it from the front end, rebuild it with a serverless architecture, and integrate them using a loosely coupled approach. Furthermore, the analysis execution area was made multi-tenant and consolidated, aiming for an environment where additional analysis algorithms developed by our company can be implemented at any time.” (Mr. Obata)

The Digital Solution Promotion Division, which began restructuring the architecture, began development in December 2020 and began providing services under the new architecture in April 2021. The customer VPC that stores user data is single-tenant for security considerations, while the VPC for the analysis environment that performs data analysis is multi-tenant. The multi-tenant environment utilizes serverless architectures such as Amazon API Gateway, AWS Lambda, AWS Step Functions, and Amazon Elastic Kubernetes Service (Amazon EKS), among others. Servers are activated only when execution instructions are sent to the API, enabling the utilization of container-based computing resources.

The material data analysis environment, revitalized by Reachitekt, is steadily gaining traction and accepting new users.

"Currently, it is primarily utilized by customers in the research departments of domestic and international chemical manufacturers. Customers who used the conventional environment were asked to gradually transition to the new environment while explaining the merits of advanced analysis algorithms and increased time spent on analysis.” (Mr. Obata)

Cost optimization was realized by migrating to a new architecture and setting the analysis environment to a serverless configuration.

“If GPU instances continue to be used for a long period of time in a conventional environment that runs constantly, things that were expensive and difficult to provide can now be serverless and cost reduced, and they can now be offered at a low cost. The time required for one analysis and analysis such as image analysis, which could not be proposed because it took tens of hours with conventional CPU-only architectures, has been shortened to a few hours and can now be proposed.” (Mr. Obata)

Regarding the development of new analysis functions within the material data analysis environment, Hitachi is adopting agile development practices. This allows them to set up a development environment for each function and monitor the functions under development in real time.

“This has improved development efficiency and contributed to enhanced service quality” (Mr. Obata)

Providing advanced analysis functions and reducing analysis time meets the demands of chemical manufacturers and others aiming to swiftly implement Proof of Concept (PoC) and deliver results, thereby enhancing customer benefits.

“We believe that the fact that many customers continue to use our solutions is proof that they are aware of the merits of using material data for analysis. Customers who co-created it have also said that experimental efficiency has improved by tens of percent. Furthermore, we have heard from customers that they are utilizing material data analysis environments not only within one department but also across research institutes, business divisions, group companies, and overseas. I feel that we have contributed to customers' digital transformation (DX) by reducing the material development period.” (Mr. Fukuoka)

The material data analysis environment continues to evolve while adding new analysis functions even after the reconstruction. Going forward, we are also considering improving usability using generative AI, expanding global expansion, and coding infrastructure (IaC) to reduce development man-hours, and we are looking forward to continued support from AWS.

“In the future, as material data is stored on the foundation, more data will be handled. To achieve this goal, significant computational resources will be necessary. Therefore, we aim to create a seamless analysis environment by continuously enhancing the architecture with various services.” (Mr. Fukuoka)