Riversoft Reduces Inferencing Time for Smart Travel System by 75% with Generative AI on AWS

Riversoft is a Taiwan-based software development company serving a broad client base in industries ranging from travel to healthcare and e-commerce. When the 2020 pandemic struck, Riversoft pivoted to invest in artificial intelligence (AI)—aiming to use AI to understand travel offerings and consumer demands, connecting these on a semantic level to present tailored travel experiences.

The company began by focusing on the Taiwan-Japan tourism corridor. Japan-focused packages are a top seller at Taiwanese travel agencies, with the country sending about 5 million visitors to Japan each year. Working with a Japanese partner, Riversoft built the Japan Boost Travel platform in 2023 to showcase local attractions to international tourists. The platform caters to both individual travelers and agencies and tour operators.

A key component of the platform is Smart Gateway, Riversoft’s proprietary “travel brain” that uses in-house ontology technology built with large language models (LLMs). Its LLMs convert free text from consumers into information that fits into a semantic deep learning structure. For example, when consumers query about specific travel experiences, such as finding tour packages for a seasonal event in a particular destination, the service recommends the most relevant products to match their requests.

In the early days of development using online AI tools, the company struggled with long inference times, often taking 20–30 seconds to yield semantic search results. The accuracy of the information was also subpar, with the LLMs frequently delivering irrelevant results. Riversoft sought an alternative generative AI provider to refine its “travel brain.”

Riversoft chose Amazon Web Services (AWS) as its technological foundation, relying on Amazon Lightsail as a virtual cloud server. Seeking to diversify its LLMs, the company began testing Claude 3 generative AI models on Amazon Bedrock.

One of its test use cases involved a complicated in-house customer-service call center that required machine learning (ML) logic. Claude Shen, chief technology officer at Riversoft, says, “When we were using our previous AI provider, we couldn’t solve the problem. But when we started testing Claude models on AWS, we realized that Claude’s logical reasoning abilities are stronger.” Riversoft currently employs Claude models on Amazon Bedrock for retrieval-augmented generation, converting customer inputs into semantic data for processing in the travel brain.

Riversoft also received support from AWS solutions architects to deploy BERT models via Amazon SageMaker as part of its Smart Gateway service. “We use BERT models for diverse activities such as named entity recognition and document classification,” Shen explains. “We need reliable NER models to help our system identify domain-specific terms or word combinations a customer may type and how those are related to available travel offerings.” For example, a user may type in a point of interest such as Tokyo’s Yoyogi Park, and Smart Gateway would then return all travel packages that include that destination.

In consultation with AWS, Riversoft also began experimenting with AWS Fargate as a serverless compute engine for its container-based architecture, in combination with AWS Lambda serverless code. “The support we received from AWS in deploying models on Amazon SageMaker made the learning curve much faster and more efficient than trying to work things out on our own,” Shen shares. “The AWS team gave us advice on how to use AWS Lambda and AWS Fargate to make our whole Smart Gateway system more efficient and scalable.” 

Riversoft launched the Smart Gateway software-as-a-service in early 2024, with Japan Boost Travel as a pilot user. Japan Boost Travel reports significant time savings since implementation. Vendors can now select from tens of thousands of products in the travel pool in real time, which helps the company increase revenue, raise awareness of lesser-known packages and destinations, and improve retention among vendors and end customers.

Kenji Fukushima, manager of the Tourism Business Planning Department at Japan Boost Travel, says, “Smart Gateway automates the product-to-user matching process, reducing the time required from an average of 4 hours to less than 10 minutes. The system also addresses the typical 80/20 distribution issue, where 80 percent of sales go to a few popular products and only 20 percent to lesser-known packages. This encourages smaller to medium-sized travel agencies to adopt Smart Gateway to gain higher visibility in the market.” Over the next two years, Japan Boost Travel aims for a 15–20 percent market penetration rate within travel agencies in both Taiwan and Japan.

Shen adds, “After fine-tuning Smart Gateway with various models and data sets, we've achieved an accuracy rate above 90 percent. Additionally, inferencing time has been reduced from 20 seconds to just 5, allowing for rapid experimentation. We've also seen a 20 percent reduction in inferencing costs by using Claude models on Amazon Bedrock.”

The company’s innovation journey continues at full speed; Riversoft is now collaborating with AWS and AWS Partners to develop a multi-model system to filter irrelevant queries and dispatch generative AI tasks to different models. Shen says, “Multi-model means multi-agent, and this is where the travel industry is headed. Our goal is to create an omnipotent agent to dynamically package and combine all travel components, such as flights and hotels, according to each user’s needs. If we can do this in real time, or even within 10 minutes, it’s still about 100 times faster than conventional service models.”