Japan’s Ministry of Economy, Trade and Industry Powers gBizINFO Corporate Information Search Database with AWS
Solving Performance Issues and Promoting Further Utilization of Data
The Ministry of Economy, Trade and Industry (METI) in Japan uses Amazon Web Services (AWS) to power its corporate data search service—the gBizINFO database. METI uses Amazon Neptune and Amazon Aurora to provide a scalable and reliable information search database for over five million corporations. As METI continues to drive towards government digitalization, the organization looks to expand their capabilities and provide catered experiences across Japan.
By adopting AWS, we have significantly improved the performance of the gBizINFO open data corporate information search service. I feel that gBizINFO, together with services related to authentication and administrative procedures, will be used as general-purpose assets in the government's promotion of digitization of administrative services regardless of field."
CIO Aide, Ministry of Economy, Trade and Industry
Converting Corporate Information to Open Data to Drive Government Digitization
gBizSTACK is a METI platform designed to promote the digitization of administrative services. METI is developing a series of services and systems covering digital authentication, administrative procedures, data linkage, data analysis, open data, and open source. Some services are already in operation and some are undergoing construction and verification in anticipation of starting full-scale operations. gBizINFO was the first to officially launch in January 2017 as a corporate information search database.
"In October 2015, it was specified that each corporation needs to have a unique corporate ID number. Later, in December 2016, the Basic Plan for the Advancement of Public and Private Sector Data Utilization was announced and implemented, thus making open data initiatives an important theme. The aim of gBizINFO is to link corporate information held by the government to a corporate ID number to better share information with the private sector," explained Yoshihiro Hosokawa, chief information officer aide, METI.
METI first launched the site on a trial basis before starting full-scale operations. As of January 2021, the database contains approximately seven million entries of corporate activity data. Searching by corporate number and name allows users to browse corporate type, location, and activity data. This includes financial information and patents, reporting and certifications, subsidies, commendations, and procurement in government offices. By using SPARQL (a computer language used in searching datasets), API, and REST (Representational State Transfer, a design condition suited to linking multiple software programs in a distributed system) API, the data is presented to users in a manageable format that can then be used by external services.
Seeking a System Configuration to Solve a Myriad of Problems
The gBizINFO system is built using a Web3 hierarchical model. From 2015 to 2018, when test operations started, the database operated using open source GraphDB software as the primary server on a different cloud infrastructure. However, Tomonari Hiroshige, head of Information Project Office, Policy Planning and Coordination Division, Commerce and Information Policy Bureau, recalled that this service caused a number of issues. The platform struggled with reliability, availability, scalability, and support. "In the database, some search conditions slowed the execution of SPARQL queries, rendering it impossible to meet the response requirements set by METI."
As a result, METI carried out a comprehensive review of the system infrastructure. This review allowed METI to start running web applications and REST API processing on RDB, and SPARQL on GraphDB in early 2019. By configuring the commercial DB graph function and RDB on separate cloud infrastructures, METI was able to resolve some issues, but not all.
"This managed to resolve a number of issues from the previous system, but some problems with reliability remained. For example, there was an inability to use a cluster configuration due to high costs, meaning we had to operate with a single instance. Also, for the database, response requirements remained unmet," said Hiroshige.
Migrating to Amazon Neptune and Amazon Aurora to Achieve Performance Improvements
Faced with continuous issues, METI reviewed their services again and selected AWS as their underlying infrastructure.
"In selection, we carried out a proof of concept and confirmed that we could expect both major improvements in performance, in addition to better reliability and operability. We received a lot of support from the AWS Professional Services technical team and appreciated the flexible cloud contract structure and its high performance," said Hosokawa.
Beginning in July 2020, METI selected Amazon Neptune and Amazon Aurora PostgreSQL for the relational database after analyzing the high-speed and high-reliability graph database available in the AWS Asia-Pacific (Tokyo) Region. By migrating to AWS, METI achieved cost savings that enabled a primary database and a replica. These two systems were placed in different Availability Zones within the same AWS Region, which increased redundancy and improved reliability.
"We have seen a dramatic improvement in response times when searching by corporate number and when acquiring detailed information. However, searching through five million corporate names in itself requires sequential searching, and thus delays are unavoidable," said Kazuki Sakai, a digital promotion manager in the Information Project Office, Policy Planning and Coordination Division, Commerce and Information Policy Bureau.
Initially, the time spent searching by corporate names was reduced from 48 seconds to 16 before tuning SPARQL queries to bring response times to a fraction of a second. "Measures such as tuning SPARQL queries gave us outstanding speed improvements, bringing the response time for searches of corporate names down to 0.1 seconds," said Sakai. Migrating to AWS created vast improvements in reliability, scalability, availability, and operability.
Looking Ahead: Utilizing and Sharing Open Data
These trends indicate that organizations are moving towards utilizing open data provided by gBizINFO, particularly in the construction of new services. Looking forward, METI will continue to improve users understanding of the corporate data it handles and will continue to expand their scope of items. "By enabling API linkage, we would also like to improve the efficiency of data collection from other government offices as well," said Hiroshige.
Hosokawa stated the overall vision for gBizSTACK, including gBizINFO, was that it could take a leading role in initiatives towards creating a digital government centered upon the digitization of administrative services and procedures.
"Although gBizSTACK itself is still closed to some corporate administrative services, I feel that gBizINFO together with services related to authentication and administrative procedures will be used as general-purpose assets in the government's promotion of digitization of administrative services, regardless of field,” said Hosokawa.
About Ministry of Economy, Trade and Industry
The Ministry of Economy, Trade and Industry (METI) was established January 6, 2001 in Japan. With a staff of over 7,000 employees, METI is a government agency for administrative tasks with jurisdiction over economic and industrial development, mineral, and energy resources.
Benefits of AWS
- Decreased response times in search requirements from 48 seconds to 0.1 seconds
- Achieved cost optimization by reducing infrastructure that generated greater reliability
- Reduced total cost of ownership for the system infrastructure operations
- Resolved reliability, availability, and scalability issues
AWS Services Used
Amazon Neptune is a fast, reliable, fully managed graph database service that makes it easy to build and run applications that work with highly connected datasets.
Amazon Aurora is a MySQL and PostgreSQL-compatible relational database built for the cloud, that combines the performance and availability of traditional enterprise databases with the simplicity and cost-effectiveness of open source databases.
Amazon CloudWatch is a monitoring and observability service built for DevOps engineers, developers, site reliability engineers (SREs), and IT managers.
AWS WAF is a web application firewall that helps protect your web applications or APIs against common web exploits that may affect availability, compromise security, or consume excessive resources.
Organizations of all sizes across all industries are transforming and delivering on their missions every day using AWS.
Contact our experts and start your own AWS Cloud journey today.