Ricoh Develops Its Own AI Model Using LLMs in Just 3 Months Using Amazon SageMaker

Having announced its goal to transform from an office automation manufacturer to a digital services provider in 2020, Ricoh launched its mid-term management strategy to increase digital services to account for 60 percent of sales by 2025. “We strive to develop and provide optimal solutions for our customers by using AI for image, voice, and natural language processing,” says Yoshiaki Umetsu, director of Ricoh’s Digital Technology Development Center, which helps set the path of the company’s digital services as part of the Digital Strategy division. Ricoh began developing AI in the 1990s and has worked on deep learning AI since 2015, applying it to visual inspections and vibration monitoring. In 2020, the company announced AI for Work, which uses a BERT-based natural language processing technology to analyze office documents and the voices of customers who call into its call centers for better work efficiency and customer assistance. However, because the BERT model has performance limitations caused by the necessity of supervised training, Ricoh then focused on large language models (LLMs) that can train on massive datasets, using an LLM-based open-source software model equivalent to GPT-3 to develop its own AI model.

“For a simple analogy, we enhanced a BERT model that has high school–level reading comprehension to a level where it can talk to people. That is our AI for Work,” says Umetsu. “Meanwhile, GPT-3 has the reading comprehension level of a new employee who graduated from a university so that it can write program code and proposal documents. We’re fine-tuning a GPT-3-comparable AI model with additional business data to raise its capabilities to the level of a practical worker.”

Ricoh aims to create a business-specific generative AI. For example, when used in multifunction printer development, it can understand that the term jam means “paper jam” rather than “confiture,” or that tray means “paper tray” and not “serving tray,” which generic generative AI cannot do. 

Ricoh adopted AWS to develop its own AI model equivalent to GPT-3. “We turned into a cloud solution to fulfill as many GPUs as required to develop an AI model within our budget and buy lead time,” says Umetsu. “However, GPU machines were hard to find anywhere in early 2022, and AWS was the only vendor that could provide many GPU instances immediately. Additionally, we weren’t familiar with large-scale, GPT-3 class AI engineering, so we asked for help from AWS.” This help was in the form of Amazon Machine Learning Solutions Lab, where experts help identify and build machine learning solutions.

Taking advantage of the AWS support program, Ricoh applied to relax the cap on Amazon Elastic Compute Cloud (Amazon EC2) P4d Instances and created a massive GPU environment. The company built a distributed learning environment using Amazon SageMaker—a service that is used to build, train, and deploy ML models for virtually any use case—as architecture to minimize learning time. The team initially stored training data in Amazon Simple Storage Service (Amazon S3)—an object storage service offering scalability, data availability, security, and performance—but during development, GPU processing speed outstripped data transfer rates, meaning training was sometimes performed with empty data or the data was lost during transfer. Ricoh solved the issue by transferring data to Amazon FSx for Lustre, which provides fully managed shared storage with the scalability and performance of the popular Lustre file system.

“We consulted experts from the ML Solutions Lab during development,” says Umetsu. “Our lack of experience and knowledge in AI development using massive amounts of GPU instances and huge storage capacities worried us. However, working alongside the Amazon Machine Learning Solutions Lab to develop the architecture, we accumulated our own AI engineering expertise and raised our skill levels so that we could solve problems in-house.”