RareJob Technologies Uses Amazon SageMaker for English Speaking Tests to Speed Up ML Model Development by More Than 10x

The RareJob Group is developing human resource education services for English, such as Smart Method, which is English conversation coaching for corporations, and RareJob English Conversation, which is an English conversation service that has 1 million members, 60 million
lessons, and 6,000 instructors.

In June 2020, the company developed the English speaking-ability measurement system PROGOS, which achieves automatic scoring using AI, and began providing the RareJob Speaking Test Powered by PROGOS (speaking test) service for individuals and corporations using PROGOS.

The speaking test measures the examinee’s ability to speak English. Examinees take the test online, and it takes about 20 minutes to answer the questions. Examinees are notified of the results in as little as 2–3 minutes, and they know their English proficiency based on international assessment standards. “The speaking test is a service released so that test-takers can grasp their own abilities and use them as a reference for their study plans,” says Kentaro Haneda, executive officer and chief technology officer at RareJob Technologies. “Scoring with AI provides a solution for evaluation blurring due to an individual rater’s experiences and senses, helping objective results to be announced quickly.”

The RareJob Group migrated its RareJob English Conversation service from another cloud provider to AWS in 2016. Since then, most of its services have operated on AWS.

“We decided to migrate to AWS because we judged that the bottleneck of our infrastructure would be a hindrance to our business,” says Haneda. “Since most of the lecturers in RareJob English Conversation are in the Philippines, we also evaluated AWS’s region in Singapore as well as its excellent support, technological development, and amount of technical information.”

PROGOS, which began development in 2019, also uses AWS. PROGOS consists of two services: the application side, which asks examinees questions and accepts answers, and the scoring side, which stores the voice data from examinees and scores it with AI. Both services use containers and serverless, managed services such as Amazon Elastic Container Service (Amazon ECS), AWS Fargate,  Amazon DynamoDB and AWS Lambda for the runtime environment. To loosely-coupling services on the application side and the scoring side, PROGOS uses Amazon Simple Queue Service (Amazon SQS).

“Since PROGOS is used for various services, we designed an architecture with scalability in mind,” says Haneda. “Also, we use Amazon SQS to achieve preprocessing launched at the same time that the user starts taking the test. Amazon SQS then quickly transitions to scoring after confirming the completion of the test.”

RareJob Technologies uses Amazon SageMaker to develop ML models for AI scoring, and AWS Glue and Amazon Athena to acquire data from examinees for model development. “At first, we were developing ML models on a local PC, but only one model could be launched at a time per developer, which created a bottleneck in development,” says Takuya Yamashiro, ML engineer in the EdTechLab at RareJob Technologies. Yamashiro and his team once tried Amazon EC2 and Amazon ECS, and they needed larger instances with a lower cost. “So, we focused on model training jobs using Amazon SageMaker. We decided to adopt it after seeing that instances with more GPUs and larger memory could be selected relatively freely and that ML could be trained at a lower cost by using things like managed spot training.”

The adoption of Amazon SageMaker significantly improved the efficiency of developing ML models. “We were able to reduce learning time by up to a quarter compared to model development using a local PC,” says Yamashiro. “It is now possible for one developer to simultaneously perform multiple learnings up to the service limit, and learning efficiency has improved by more than 10 times.”

The RareJob Group is also accelerating its use of AWS in terms of business. Employees transitioned to remote work due to the COVID-19 pandemic, and when the capacity of existing VPNs was insufficient, the company adopted AWS Client VPN in a short period of time to create a remote work environment for approximately 150 employees. The company also began using Amazon WorkSpaces for department personnel that handle sensitive management information and personal information. For the routine work of retrieving necessary data from cloud services and uploading files to document management tools, the company developed a system that automatically acquires data using AWS Lambda, reducing the workload by approximately 100 hours per month. “The mission of RareJob Technologies is to increase the value of the RareJob Group, and AWS has established a style that leads to solutions for issues while actively uncovering potential needs,” says Katsuaki Hirakawa, IT solution manager at RareJob Technologies.

The company also set up a department specializing in DevOps to improve development productivity as well as built a continuous integration and delivery environment to support daily infrastructure operations.