NTT DOCOMO's base station steel tower rust inspection tool built with AWS has reduced costs and improved safety.

NTT DOCOMO, INC. ("NTT DOCOMO"), one of Japan's largest mobile network operators, is required to inspect approximately 50,000 base station steel towers for rust and deterioration every few years in order to perform maintenance and ensure network access for users. However, it is dangerous and inefficient for inspection crews to climb towers that are dozens of meters high, so NTT DOCOMO decided to develop a unique system that uses drones to photograph and inspect towers.

To make this system work, it was necessary to combine artificial intelligence (AI) and machine learning (ML) technologies with computing power to develop an image recognition system that would analyze thousands of photos taken by drones to detect rust. So, we used Amazon Web Services (AWS) to quickly deploy a cost-effective solution to ensure safe and efficient steel tower inspections and provide stable network access to our users.

Innovation is a specialty of NTT DOCOMO, which serves approximately 73 million users with its LTE network and one of the most advanced LTE-Advanced networks in the world. NTT DOCOMO is a leading global developer of 5G networks, short-range wireless infrastructure, and emerging IoT solutions. We have a long history of building on AWS and have migrated some of our own web services systems, data analytics platforms, and data warehouses to AWS. NTT DOCOMO has also developed its own drone management system called Docomo Sky, which is used with AWS. Therefore, it was a natural choice to use AWS for the project to automatically inspect base station steel towers for rust using images captured by drones. "We knew from previous experience that AWS would be very useful for software development and data management, so we didn't want to consider on-premises servers or another management system," said Mr. Issei Nakamura, AI specialist at NTT DOCOMO.

Rust and deterioration of steel towers is a serious problem. "If left too long, rust can reduce the availability of the network." But visual inspections by workers in the field can be dangerous. ”Since climbing up and down steel towers is hard work, we decided to use drones to inspect it to ensure the safety of the facility and the workers. At the same time, our goal was to standardize inspection results and improve inspection accuracy,” said Mr. Nakamura.

In 2017, NTT Docomo began using drones to take pictures of steel towers. Since it is time-consuming for staff to sift through thousands of images, we decided to develop an image recognition system using AI and machine learning technology. ”We consider data to be an extremely valuable asset, which is why we developed our own image recognition system.”  

The NTT Docomo team developed two prototypes of the rust detection system in about a month. In addition to the low cost of prototyping, the team had access to a wealth of information about AWS services. The system was released internally in January 2020. NTT DOCOMO began by training a model to detect rust and damaged areas using about 100 images, but soon it realized it was running out of images and began continuously collecting data to train the model. We are currently using this system to inspect base station steel towers in our own network. Inspections are performed using a two-step process. First, the AI inspection does rust detection. The next step is for the operator to perform a rigorous inspection, if necessary.

The recognition system processes approximately 400 images per steel tower. Each image is approximately 7 MB in size and 5,472 x 3,648 pixels. NTT DOCOMO initially used an Amazon Elastic Compute Cloud (Amazon EC2) P2 instance for inference, but it switched to an Amazon EC2 G4 instance in August 2019. The G4 instance is powered by an NVIDIA T4 Tensor Core GPU and is twice as fast as the P2 instance at less than a quarter of the cost. Estimated time per image is 5 seconds versus 10 seconds, and cost per hour is 0.71 USD versus 1.542 USD. "The Amazon EC2 G4 instance has exceeded our expectations," said Mr. Nakamura.

To further reduce costs, NTT DOCOMO uses AWS Batch. This service dynamically provisions the optimal number of Amazon EC2 G4 instances based on the volume and resource requirements of submitted batch jobs. "AWS Batch is the most important management service in our system," continued Mr. Nakamura. "Using AWS Batch, we were able to significantly reduce operating costs. We don't want to keep an Amazon EC2 G4 instance running all the time because base station steel tower inspections are not a daily task." Amazon API Gateway and AWS Lambda are used to receive HTTP requests. The code is then executed to store these HTTP requests in Amazon Simple Storage Service (Amazon S3) and submit the job to AWS Batch. “Using Amazon S3, we can collect images very easily," said Mr. Nakamura. Previously, we had an internal data warehouse for image sharing. "It allows us to train our models more efficiently.”

Currently, 15 tasks can be performed simultaneously by the image recognition system. With AWS Batch, NTT DOCOMO can easily increase this number. In addition, inspection results can be standardized by comparing images from different steel towers and by comparing images from different collection periods. NTT DOCOMO expects that the use of this system to quickly locate and repair damaged steel towers will contribute to the long-term stabilization of network quality for end users and increase customer satisfaction. Employee safety has certainly improved with the switch from visual inspections to drone inspections. Since switching to drone surveillance, there has not been a single incident related to the inspection of base station steel towers. The new system has also improved the efficiency of steel tower inspections and significantly reduced costs. Compared to visual inspection, the inspection time per steel tower was reduced by approximately 100 minutes. “In 2020, we plan to use drones to inspect about 1,500 steel towers. This will save more than 2,000 hours," said Mr. Nakamura. “This translates into a cost savings of more than 100,000 USD."

New business opportunities may arise for NTT DOCOMO with the success of the drone inspection program. "We believe that such inspection data will be a valuable asset," said Mr. Nakamura. "Based on our experience with base station steel tower inspections, we plan to develop inspection services for other infrastructure, such as bridges and buildings."

By using AWS to move from a visual inspection protocol for towers to an AI-powered drone inspection system, NTT DOCOMO not only increased employee safety and reduced costs, but also created a new revenue stream opportunity. In addition, by reducing rust and damage to steel towers, Japan's largest mobile operator can devote more time and resources to providing the best mobile services to its users.