Karakuri’s AI powers Japanese customer service using AWS Trainium

Karakuri provides AI-powered chatbots and customer support automation tools to Japanese enterprise companies, handling thousands of customer interactions daily. “Japanese customer support often requires a higher standard of service compared with other countries,” says Tomofumi Nakayama, chief product officer at Karakuri. “Even minor mistakes or honorific language errors can lead to unhappy customers. To address this, we needed to develop an LLM specifically focused on the Japanese language and Japan’s customer support culture.”

As Karakuri’s models grew larger and became more sophisticated, the company began to encounter obstacles. Training costs on GPU-based systems became prohibitive, and securing sufficient GPU capacity proved extremely difficult amid growing industry demand. For a small business with limited resources, these constraints highlighted the need for more efficient, scalable infrastructure to continue innovating at a critical growth phase.

Since its founding in 2016, Karakuri has relied on AWS for its core infrastructure. AWS continually innovates in purpose-built chips and advanced infrastructure to build its portfolio of compute solutions. This portfolio would provide Karakuri with the enterprise-grade technology that it needed to train increasingly sophisticated language models at scale without prohibitive costs. In 2023, when AWS Japan launched an LLM development accelerator program, Karakuri was selected to participate—a decision that would change how the company approached model training and development.

Karakuri’s initial goal was to create Japan’s highest-performing, most accurate LLM. The team began using AWS Trainium, a family of AI chips purpose-built by AWS for AI training and inference to deliver high performance while reducing costs. Although Karakuri was initially cautious about adopting new infrastructure, AWS provided onboarding support and training through hands-on sessions and detailed examples that accelerated the team’s progress.

The company architected its training environment using AWS ParallelCluster, an open source cluster management tool, to orchestrate distributed learning across multiple AWS Trainium instances. To maintain reproducibility and consistency, Karakuri coded its entire environment using AWS CloudFormation, which speeds up cloud provisioning with infrastructure as code. Using the service, the team reliably replicated configurations as they experimented with different model architectures. In just 2 months, full-scale training was underway on LLMs trained specifically on extensive Japanese customer support data.

By adopting AWS Trainium, Karakuri reduced its training costs by over 50 percent. These substantial savings empowered Karakuri to create Japan’s most accurate Japanese language model while staying well below budget. Beyond cost savings, migrating to AWS Trainium removed the capacity constraints that had previously hampered Karakuri’s development timelines.

“For startups with limited resources, costs often become a significant issue when using large-scale models,” says Yuki Yoshida, researcher at Karakuri. “Using AWS Trainium empowers us to use the latest large-scale models, which creates a substantial competitive advantage.”

The infrastructure stability also delivered unexpected productivity gains. With consistent resource availability and minimal hardware issues, Karakuri’s team could focus on innovation rather than troubleshooting. The company also used AWS Inferentia—designed by AWS to deliver high performance at the lowest cost in Amazon Elastic Compute Cloud (Amazon EC2) for deep learning and generative AI inference applications—to serve the models. This helped reduce latency, and customers responded positively to the improved response speeds.

Looking ahead, Karakuri has already begun training vision-language models and plans to develop multimodal models that handle text, images, and audio. “Using AWS, our reliability has improved, and with support in acquiring new customers, we’ve generated higher revenues and are able to expand our market share,” says Yasuhisa Nakashima, researcher at Karakuri.