LUCI Sky from Lenovo delivers analytics at lightning speed with Talend and AWS
Talend is an AWS Advanced Technology Partner
Hardware and PC giant Lenovo was experiencing explosive business growth and exponential increases in data volume. The company knew that supporting analytics solely on premises would not be sustainable, and it decided to move to the cloud. With Amazon Web Services (AWS), Talend, and other partners, Lenovo built a hybrid-cloud platform for analytics. When the data volume ballooned to unimagined heights in 2018, the company again turned to Talend for help. Talend enabled Lenovo to decrease the data size by 40 percent, accelerating analytics speed, and almost doubled weekly deployments.
Lenovo was essentially a victim of its own success. The business had really taken off in the 2000s. Lenovo was trying to use data to understand its customers, but the massive volume and manual processes were making that difficult. The company’s original customer data analytics platforms, Microsoft Excel and Microsoft Access, were straining under the weight. It was time for a change. Lenovo moved to MySQL on AWS, marking the start of its cloud journey.
Time passed, and Lenovo’s conglomeration of on-premises, cloud, and SaaS technologies began to affect data connectivity. It was time for the next step in the cloud journey—a customer intelligence platform with a hybrid architecture (AWS and Lenovo servers)— that would require extract, transfer, and load (ETL) functions for 250 TB of data. The platform was built and quickly became a core part of the business, delivering critical intelligence to all parts of the company. But just five years later, the size of the data in the platform was fast approaching 1 PB. Lenovo knew that this could have a major impact on the system.
In 2013, with AWS, Talend, and other partners, Lenovo built Lenovo Unified Customer Intelligence (LUCI) Sky, an AWS-based platform that runs the analytics workload for the entire Global Analytics & Operations (GAO). LUCI Sky’s tech stack consists of AWS services such as AWS Glue and Amazon Elastic MapReduce (Amazon EMR) for advanced analytics, Amazon Redshift and Amazon Relational Database Service (Amazon RDS) for database services, Amazon Simple Storage Service (Amazon S3) and Amazon Elastic File System for storage, and Amazon Elastic Cloud Compute (Amazon EC2) for compute. For data integration and data quality, Lenovo chose Talend Data Fabric.
A few years later, Lenovo wanted to reduce analytics delivery time and build more dashboards to increase the value of LUCI Sky for sales, demand generation, and inventory management. And, the company turned to Talend and AWS again. With AWS Lambda, Amazon S3, Amazon S3 Glacier, and Talend Data Fabric, Lenovo used a 30- 60-90-day approach to move data to different types of storage depending on age and
“We chose AWS because not only was it cheaper, but it was the most effective solution for what we needed in a cloud platform.”
- Ranadip Dutta, Solution Architect Manager, Lenovo
Results and Benefits
Lenovo credits LUCI Sky for most of its business success since 2013. Through analytics and dashboards, LUCI Sky enabled Lenovo to better understand customer preferences, and by 2016, each Lenovo retail business unit had achieved revenue growth of 11 percent. In addition, LUCI Sky had cut analytics delivery time on 600 TB of data from two weeks to one week. But the major benefits kicked in after the data cleansing project. Analytics SLAs dropped from weekly in 2017 to six times a day in 2019, with the promise of real-time SLAs in 2020. New requests are acknowledged in an hour. Deployments have increased from five to seven a week to 12 or more. Lenovo went from saving 70-80 hours to more than 100, error free. Plus, aged tickets stand at less than 1 percent.
“I see us being able to meet SLAs for real-time delivery very soon. Data reduction with Talend will give us that.” — Ranadip Dutta, Solution Architect Manager, Lenovo
“Talend gave us the real automation we needed, along with collation, massaging, cleansing, scheduled jobs and more. The execution was really smooth.”
- Ranadip Dutta, Solution Architect Manager, Lenovo
Talend Makes It Easy for Anyone to Become a Data Expert
After Lenovo chose Talend for data integration and quality, the company quickly discovered Talend was easy to learn and use. Lenovo is a fast-paced company with lots of moving parts and pieces, and it doesn’t have time for long learning curves or failure. Even people who are not completely familiar with building data integration pipelines can become proficient with Talend in a month. “It delivered the productivity we needed,” said Ranadip Dutta, Solution Architect Manager at Lenovo. “When we needed help with our data reduction and cleansing project, our experience taught us that Talend was the right solution for that project.”
Diamond-Level Scalability: No Limits for LUCI Sky with AWS
Because Lenovo is committed to being data driven to the core, the company needs fast, flexible, and scalable analytics. Teams must be able to build dashboards quickly that can satisfy the needs of sales, inventory management, and demand generation. For example, sales teams need to be able to drill down to a particular region or shop for information about sales patterns. They also want to predict sales and trends six months into the future. With AWS, they get scalability, availability, latency, elasticity—and more. “The world is changing. There’s multicloud and Hadoop. AWS solutions give us the modular architecture that delivers flexibility along with scalability,” said Dutta.
Extraordinary Performance and ROI—All in a Day’s Work
At Lenovo, waiting around for answers is not an option. Since reducing data by 40 percent, LUCI Sky’s performance has met all SLAs. It ingests over 40 GB from more than 60 data sources each day. It runs 4,000+ processes, turns tickets around in less than three hours, and its compute is 800 cores. And with all that, it still delivers analytics every four hours or so, feeding dashboards and other intelligence tools. LUCI Sky has also delivered on its annual goal of 10X return on investment.
Talend Data Fabric enables organizations to ingest and process data at scale anywhere—in the cloud, on premises, or on a hybrid infrastructure.
AWS offers a broad set of global cloud-based products including compute, storage, databases, analytics, and networking.
Talend is a leading data integration and data integrity company. In a single platform, the company brings together the capabilities that ensure enterprise data is complete, clean, compliant, and readily available to everyone who needs it.
Published October 2020