Elevenia Instills a Data-Driven Culture with AWS Analytics Platform
Many organizations aspire to be data-driven, but few understand how to do it. In 2016, the leaders of e-commerce site Elevenia had a big idea: to instill a data-driven culture, it would build a new, cloud-based analytics platform fueled by real-time business intelligence (BI). Elevenia launched in 2014 in Indonesia as a customer-to-customer (C2C) online marketplace. In 2018, the site had more than 4.8 million listed products and an average of 13,000 transactions a day.
To carry out its idea, Elevenia formed a data management team including a data scientist, data engineers and BI analysts. The team would directly contribute to product development by integrating analytics in the early stages of the product development lifecycle. Elevenia chose to build its dedicated BI infrastructure on the Amazon Web Services (AWS) Cloud, because one of its parent company’s businesses was already successfully running on AWS. Priorities for the new infrastructure included capacity to scale, value creation with minimal operations effort, and at least 95 percent uptime.
“With the AWS Cloud, we can build more complex analytics models that scale.”
Rangga Sobiran, Head of Data Architecture and BI, Elevenia
AWS Services Used
Elevenia is a customer-to-customer (C2C) online marketplace based in Indonesia. Launched in 2014, the site has more than 4.8 million listed products and averages 13,000 transactions a day.
- Saves almost $20,000 monthly by building analytics tools in-house
- Keeps operations overhead low with automation and managed services
- Facilitates learning and broadens skill sets for data management team
- Offers flexibility and agility to support deep analytics models
- Improves product quality with machine learning
AWS Services Used
From On Premises to On Demand
Until 2016, Elevenia had been running all operations and some analytics on premises using Oracle servers. “Because operations and BI shared server resources, analysts had to be conservative in running queries, especially deep analytics,” says Rangga Sobiran, head of data architecture and BI at Elevenia. “Latency was high, causing delays in returning queries, and we had to avoid downtime to the core business.”
“Now we have flexibility in terms of how we want to achieve transformation in the data warehouse,” Rangga says. “With the AWS Cloud, we can build more-complex analytics models that scale. Because Amazon Redshift is made for analytics types of queries, the performance is better than anything we’ve experienced.” The team has also benefited from the ease of interface with other AWS services such as Amazon Kinesis, which is used for clickstream architecture.
The diagram below illustrates the AWS infrastructure for clickstream pipeline at Elevenia:
Containers Ease Deployment
Shortly after establishing the data management team and infrastructure, Rangga put them to the test. Elevenia had been paying a software as a service (SaaS) provider to run the recommendation engine on its shopping site. Because the new team had some limited experience using the Python programming language, they decided to try building their own engine to compare cost and performance. Being on the AWS Cloud facilitated experimentation, and team members were eager to try new things.
“It’s difficult for us to get things right at the start, so there’s a lot of trial and error involved. This works very well in the cloud because we can create a proof of concept and run it for two weeks, and if doesn’t work, we can strike the idea out and try again,” Rangga says.
To simplify the deployment process for the new recommendation engine, the team set up Docker containers using Amazon Elastic Container Service (Amazon ECS). “In the long term, we felt that containers would make it easier for us to deploy the engine everywhere, even at scale,” Rangga says. Engineers use Amazon Fargate with Amazon ECS to automate deployment of containerized services, including clusters on virtual machines. “We really love this serverless concept because it requires minimal operations maintenance,” he adds.
After developing the recommendation engine, Elevenia compared the click rate from its homegrown engine with that of the engine provided by the SaaS vendor. Although both the click rate and conversion percentage increased by just 0.1 percent with the homegrown model, Elevenia is saving $5,000 in monthly fees by using the SaaS vendor. “It’s very significant for us, because basically we’re paying much less but getting the same return,” Rangga says.
The company has also notched substantial savings by stopping its subscription to Google Analytics Premium for tracking data in real time. Instead, it has adopted the Snowplow open- source technology, which is configured to run on top of Amazon Kinesis. With Amazon Kinesis and Amazon S3 storage supporting the Snowplow clickstream pipeline, Elevenia is now paying less than 2 percent per month compared to what it used to pay for Google Analytics. This brings its total monthly cost savings to almost $20,000 with the AWS Cloud and open-source products.
New Machine-Learning Models
With its recommendation engine done and analytics platform in motion, the data team is focused on improving product quality. They are building a machine-learning (ML) model using Amazon SageMaker and Amazon Rekognition to assess the quality of pictures uploaded by vendors on the Elevenia shopping site. Sellers often upload low-resolution images or those with watermarks of competing online marketplaces. When completed, the ML model will assess each image and alert sellers when images are of poor quality.
Elevenia is transforming its business with the analytics platform, both in terms of cost and human resources. “Using the AWS Cloud makes it easy for my team to expand their skill set,” Rangga explains. “For people with traditional data warehouse backgrounds, coding can be pretty daunting. But with the AWS Cloud, we can save a lot of time, and it eliminates the need to learn everything from scratch.”
Feedback has also been positive from the data team, who are motivated to explore new features on AWS and learn with online videos and documentation. “It’s been a very interesting journey,” concludes Rangga. “We constantly have new ways of experimenting, and it’s been really fun.”