Royal FrieslandCampina Builds Global Data Platform on AWS

Royal FrieslandCampina reduces the costs of a big data analysis infrastructure by switching from Hadoop to AWS Lambda. The Dutch company is a world leader in the production of dairy-based products. Royal FrieslandCampina uses Amazon Redshift to support data-driven decision making across the enterprise as well as AWS Lambda for processing tasks, including big data analysis.

Royal FrieslandCampina (RFC) is one of the world’s largest dairy companies, with approximately $12.8 billion in annual revenue. The Dutch business produces and sells dairy-based beverages, cheeses, desserts, and infant and sports nutrition products across Europe, Asia, and Africa.

Since 2015, the firm has been migrating its applications to Amazon Web Services (AWS) with the support of AWS Advanced Consulting Partner TerraAlto. For better decision making, RFC built a data platform called SourceOne on the AWS Cloud. The platform stores and processes internal data from sources such as SAP, and external data from sources including social media websites and point-of-sale systems.

“We want to become a data-driven organization,” says, Maarten Vlemmings, manager of SourceOne and global data services at Royal FrieslandCampina.

SourceOne is based on an Amazon Simple Storage Service (Amazon S3) data lake. Amazon Redshift provides the platform with a petabyte-scale data warehouse, and Amazon Redshift Spectrum allows data scientists to run queries directly against data in the data lake. Using data warehouse tools built by TerraAlto, RFC optimized a three-node Amazon Redshift cluster that was serving 50,000-plus data queries daily.

The cluster can now handle more than 100,000 queries a day. Anton Kuiper, global business intelligence and analytics architect at Royal FrieslandCampina, says, “We made the improvements without changes to the services.”

The company uses AWS Lambda, a serverless compute service, to support hundreds of functions, including system and development operations, Center for Internet Security benchmarking, and data pipeline construction. Recently, the IT team replaced a Hadoop based infrastructure analyzing campaign data from Facebook and Google Ads with AWS Lambda.

Downloads of around 300 megabytes from the two sources are stored in Amazon S3 buckets. AWS Lambda functions then quality check and deduplicate the data, preparing it for analysis in Amazon EMR. Says Kuiper, “As a result of AWS Lambda, the IT team cutmonthly costs for the Hadoop-based infrastructure from $12,900 to $1,300—a reduction of approximately 90 percent—and increased the elasticity of the infrastructure’s compute power.”