Japan’s Ministry of Economy, Trade and Industry Powers gBizINFO Corporate Information Search Database with AWS

The Ministry of Economy, Trade and Industry (METI) in Japan uses Amazon Web Services (AWS) to power its corporate data search service—the gBizINFO database. METI uses Amazon Neptune and Amazon Aurora to provide a scalable and reliable information search database for over five million corporations. As METI continues to drive towards government digitalization, the organization looks to expand their capabilities and provide catered experiences across Japan.

gBizSTACK is a METI platform designed to promote the digitization of administrative services. METI is developing a series of services and systems covering digital authentication, administrative procedures, data linkage, data analysis, open data, and open source. Some services are already in operation and some are undergoing construction and verification in anticipation of starting full-scale operations. gBizINFO was the first to officially launch in January 2017 as a corporate information search database.

"In October 2015, it was specified that each corporation needs to have a unique corporate ID number. Later, in December 2016, the Basic Plan for the Advancement of Public and Private Sector Data Utilization was announced and implemented, thus making open data initiatives an important theme. The aim of gBizINFO is to link corporate information held by the government to a corporate ID number to better share information with the private sector," explained Yoshihiro Hosokawa, chief information officer aide, METI.

METI first launched the site on a trial basis before starting full-scale operations. As of January 2021, the database contains approximately seven million entries of corporate activity data. Searching by corporate number and name allows users to browse corporate type, location, and activity data. This includes financial information and patents, reporting and certifications, subsidies, commendations, and procurement in government offices. By using SPARQL (a computer language used in searching datasets), API, and REST (Representational State Transfer, a design condition suited to linking multiple software programs in a distributed system) API, the data is presented to users in a manageable format that can then be used by external services. 

The gBizINFO system is built using a Web3 hierarchical model. From 2015 to 2018, when test operations started, the database operated using open source GraphDB software as the primary server on a different cloud infrastructure. However, Tomonari Hiroshige, head of Information Project Office, Policy Planning and Coordination Division, Commerce and Information Policy Bureau, recalled that this service caused a number of issues. The platform struggled with reliability, availability, scalability, and support. "In the database, some search conditions slowed the execution of SPARQL queries, rendering it impossible to meet the response requirements set by METI."

As a result, METI carried out a comprehensive review of the system infrastructure. This review allowed METI to start running web applications and REST API processing on RDB, and SPARQL on GraphDB in early 2019. By configuring the commercial DB graph function and RDB on separate cloud infrastructures, METI was able to resolve some issues, but not all.
"This managed to resolve a number of issues from the previous system, but some problems with reliability remained. For example, there was an inability to use a cluster configuration due to high costs, meaning we had to operate with a single instance. Also, for the database, response requirements remained unmet," said Hiroshige.

Faced with continuous issues, METI reviewed their services again and selected AWS as their underlying infrastructure.

"In selection, we carried out a proof of concept and confirmed that we could expect both major improvements in performance, in addition to better reliability and operability. We received a lot of support from the AWS Professional Services technical team and appreciated the flexible cloud contract structure and its high performance," said Hosokawa.

Beginning in July 2020, METI selected Amazon Neptune and Amazon Aurora PostgreSQL for the relational database after analyzing the high-speed and high-reliability graph database available in the AWS Asia-Pacific (Tokyo) Region. By migrating to AWS, METI achieved cost savings that enabled a primary database and a replica. These two systems were placed in different Availability Zones within the same AWS Region, which increased redundancy and improved reliability.

"We have seen a dramatic improvement in response times when searching by corporate number and when acquiring detailed information. However, searching through five million corporate names in itself requires sequential searching, and thus delays are unavoidable," said Kazuki Sakai, a digital promotion manager in the Information Project Office, Policy Planning and Coordination Division, Commerce and Information Policy Bureau.

Initially, the time spent searching by corporate names was reduced from 48 seconds to 16 before tuning SPARQL queries to bring response times to a fraction of a second. "Measures such as tuning SPARQL queries gave us outstanding speed improvements, bringing the response time for searches of corporate names down to 0.1 seconds," said Sakai. Migrating to AWS created vast improvements in reliability, scalability, availability, and operability.

Increasing demand for gBizINFO has caused the type of inquiries to change over time. Yūki Oikawa, is responsible for systems operation in the Information Project Office, Policy Planning and Coordination Division, Commerce and Information Policy Bureau. Oikawa said, "Up until 2019, most of the inquiries were about the content of the posted data. From the start of 2020, we have seen a noticeable increase in questions regarding the actual use of the data in external services, such as how to use data from gBizINFO in their own system or services and terms of use for the data."

These trends indicate that organizations are moving towards utilizing open data provided by gBizINFO, particularly in the construction of new services. Looking forward, METI will continue to improve users understanding of the corporate data it handles and will continue to expand their scope of items. "By enabling API linkage, we would also like to improve the efficiency of data collection from other government offices as well," said Hiroshige.

Hosokawa stated the overall vision for gBizSTACK, including gBizINFO, was that it could take a leading role in initiatives towards creating a digital government centered upon the digitization of administrative services and procedures.

"Although gBizSTACK itself is still closed to some corporate administrative services, I feel that gBizINFO together with services related to authentication and administrative procedures will be used as general-purpose assets in the government's promotion of digitization of administrative services, regardless of field,” said Hosokawa.