Premier Foods Ramps Up Decision Intelligence to Process 25 Million Records a Day with Pyramid Analytics on AWS

Premier Foods is home of the UK’s most-loved food brands. Found in 93% of British households, it has over 4,000 employees and operates from 15 locations in the UK. To fulfil its data analytics requirements— just one of which is processing 25 million records a day—it relied on an SAP-based data analytics system for everything from supply chain to commercial functions and sustainability. However, lowered business efficiency and productivity, caused by a rigid system with limited visualization capabilities, led it to look for more flexible data sourcing, faster processing, and more frequent reporting. It also wanted to take advantage of more data science techniques than its extant system provided. Turning to AWS Partner Pyramid Analytics, brought it measurable increases in processing speed, deepened visibility, and resource savings, within a few months of implementing the solution.

The existing Premier Food's analytics BI tool relied to a great extent on spreadsheet-based reporting and outdated data warehousing. This caused several issues stemming from lack of external data integration capabilities and built-in data visualization. A key challenge was the need for manual, offline reporting caused by lack of access to updated master data. These factors impacted business directly by limiting and slowing BI. Additionally, compliance with the recent Environmental Social and Governance (ESG) legislation in the UK meant bringing together data sets that weren’t supported by its current system.

Pyramid Analytics provides a pioneering decision intelligence platform. According to Deepa Sankar, Vice President of Product Marketing at Pyramid Analytics, , “Pyramid Analytics uniquely combines data prep, business analytics, and data science all within one platform with a super-fast direct query engine and an AI-guided, governed self-service experience to speed up insights for non-technical people." For Premier Foods, this was key to fulfilling its requirements. Additionally, with its main source of transaction data—like sales records—coming in from SAP, the new system also needed to connect with SAP Business Warehouse (SAP BW). Daniel Flynn, Analytics Manager at Premier Foods, , said, “Other solutions in the market required us to extract data and effectively copy it. That is not a solution for us, that's a work around. Being able to connect into SAP was a huge win for us and to be able to pull that data through, with prompts on our live connection, was great.”

Premier Foods uses Amazon RedShift for reporting on data other than transactions, such as core product information, from multiple sources like SAP, Hamilton Grant, and manual files. RedShift unloads that data into Amazon Simple Storage Service (Amazon S3) for future reference and analytics. Amazon S3 is also used to store data written back from Pyramid models. Amazon CodeCommit manages the code in Matillion, the Extract Transform Load (ETL) tool that the solution runs. Pyramid Analytics then enables Premier Foods to report on the data from both SAP BW and RedShift, while Amazon Cloud Watch provides alerts for ETL and Pyramid instances from a monitoring perspective.

For the implementation, the Pyramid team worked closely with Flynn’s team during the Proof of Concept (POC) stage to create a robust connection between different tools to any data source that Premier Foods needed. Further, their recommendations on server sizing, and production and development environment, helped Premier Foods to successfully structure the capacity required for the new solution.

Assessing recyclability of plastics and packaging generated is critical not only to Premier Food’s ESG compliance and the reporting required by UK’s new Plastic Packaging Tax, but also to meet the organization’s own sustainability goals. The new Pyramid-AWS model has replaced months of manual effort in compiling and processing data from multiple sources like SAP, Hamilton Grant (specification system), and manual files to report on every piece of packaging used by type. Scoping is currently in process for reporting and analysis on other sustainability aspects like supply chain tracking and energy consumption to determine the company’s carbon footprint and other environmental impacts.

It’s early days yet for Premier Foods’ use of Pyramid, with implementation being fully realized only early this year. However, Flynn is confident that as the solution gains traction, it will help the company develop a suite of reports that will meet most organizational needs. As Premier Foods moves towards the future, it will inform business decisions and course corrections to meet their goals.