C&A Anticipates Demand, Plans Store Inventory Using Machine Learning on AWS

Operating in Brazil since 1976, C&A, with its 288 stores, four distribution centers and around 15,000 associates is one of the largest retail chains in the country. To guarantee the
distribution and availability of about 200 product categories in all stores, the chain has started using machine learning in the cloud to make distribution and consumption projections. As the project gained scale, execution time became a problem, so C&A migrated the project to AWS to meet new speed and performance needs.

As one of the largest retail chains in the country, C&A has as one of its main processes the correct distribution of its thousands of products to each of its 288 stores. For products to be available for purchase, it is necessary to forecast the demand for each color, type, and size, and distribute them according to their stock levels.

C&A Advanced Analytics senior manager Caio Momesso recalls the original distribution process was defined through simple rules and manual adjustments, conducted by a team dedicated to the subject. Then, “this demand forecasting process was then reconstructed by C&A's team of data scientists and started considering seasonality, regional effects, product attributes, and stock levels using machine learning algorithms that process millions of data points in minutes, testing scenarios and reaching the best result," Momesso explains.

The chain works with about 2,000 new Stock-Keeping Units (SKUs) per week. These are products that need to be bought, sent to central stock, and then distributed to stores. “Furthermore, the projection needs to consider the product's life cycle, usually 8 weeks,” Momesso says, remembering that any mistake can cause a large impact, either by the lack of the product or by its excess, which would force the store to make discounts.

To improve the process and make it more accurate, C&A decided to invest in a machine learning solution. Its development began with the internal team that, after creating a model, carried out a proof of concept, initially running on-premises. With the growing need for more powerful processing capabilities, Momesso’s team decided to search for a cloud solution, initially choosing an AWS competitor.

Bruno Lourenço, responsible for data engineering in the analytics team, comments: “With the previous cloud solution, it was necessary to rewrite the entire model code to expand the parallelization capacity. We achieved a good result in terms of execution time, but managing the operating costs of the cluster proved to be a challenge—we couldn't accurately track the amounts that were charged. As we increased the volume of work, the cost grew exponentially."

As C&A distributes more than 200 product categories, the team initially implemented in the solution little by little. In early 2019, when 57 categories were being analyzed, the data team began to feel growing pains. “The process started slowing down. We saw that the solution worked well as a pilot, but it was not yet scalable,” says Momesso. “This worried us since we knew that the environment would grow a lot and we would need to redesign the whole solution. This was the watershed moment in deciding to change providers,” adds Lourenço.

At that time, C&A already operated its ecommerce platform on AWS and, according to Lourenço, there were employees on the team existing experience building on AWS. "With that we felt more confident that the AWS would be a good alternative to test," recalls Lourenço.

To guide the migration, C&A had a long-term vision: it was necessary to solve the immediate processing time issue and also to create an environment that could be expanded in the future. Momesso stresses that the investments made by AWS into its machine learning solutions were also decisive for their choice.

All in all, the migration process took 3 months. “It was necessary to transcribe the environment and rethink the model in order to achieve lower costs and better processing,” says Lourenço. For a while, the AWS solution worked in parallel with the existing solution, to help provide benchmarks. "When we got comfortable in relation to costs and results, in February of 2020, we turned off the previous provider,” Lourenço says.

Lourenço explains that the execution strategy was built on several AWS services, with the advice of AWS Professional Services. "Amazon SageMaker is the main one, but we ended up connecting several pieces to create a complete pipeline," he says. In the current structure, SageMaker is the working table of C&A’s data science team, which has lended more agility and scalability to the process.  "Nowadays it's very easy increasing or reducing the scale of the models when necessary," he notes.

Another solution C&A uses is Amazon CloudWatch, which works as a monitoring and observability service. AWS CodeCommit is a fully-managed source control service, a feature the team had not previously relied upon but that makes collaboration easier. Code is run through AWS Lambda, which creates calendar data frames, and data input and output starts and ends in Amazon Simple Storage Service (Amazon S3) buckets. The entire production environment is connected via AWS Direct Connect and coordinated by AWS Step Functions. Lourenço also highlights the use of AWS CloudFormation and Amazon Elastic Container Registry (Amazon ECR) solutions. With this configuration, C&A has today about 300 machine learning models to make the calibration and about 50 models to make the prediction.

Lourenço explains that when the migration to AWS was made, C&A chose to keep the codes in Python. “For that, we paralyzed executions, creating copies of models that received different input fractions and, further on, generated a single input. Inside the model we also use the thread halt feature. We are using the most of the possibilities of parallelism to save time. With this, we can correct eventual errors just in the location where they occur," he highlights.

When the migration was completed, Momesso remembers that the first major benefit was seen in dataprocessing time. He explains that the challenge was to process all  200 product categories in 7 hours, at most. In the previous cloud, the 57 implemented categories were processed in 11 hours. "There was the challenge of doing this in a smaller volume of hours. Today, we can process all 200 product categories in less than 3 hours," he says.

Lourenço also highlights a reduction in the number of errors in the data, noting that the team now has granular control over the data input. "We were able to make a prior validation to see if the necessary inputs considered the minimum requirements to be used. We didn't have this pre-check before. Amazon CloudWatch gives us an exact reading of what has happened," he explains.

Other benefits pointed out by executives are a higher level of control of both development and production environments and more accuracy in cost control, with accurate information on the cost of generating algorithms. With that, Momesso says, "Accuracy errors have been reduced by 61 percent, and data processing costs have been reduced by 4-5 times."

Momesso remembers that the previous model couldn't capture regional seasonality, such as the São João festivities in the Northeast of Brazil, and stores were short of supplies in some product categories. “Since the adoption of the new model, these stores no longer go through this. The previous model also had problems predicting product demand in smaller stores, which have more volatile behavior. After the new model implementation, small stores were the ones that benefited the most, with the greatest increases in product availability and sales growth."

Currently, 75 percent of the fashion products sold by C&A are distributed using its machine learning solution. Momesso explains that the idea is to expand the predictive model to guide the development of new products, indicating the attributes of higher demand, such as “pants with high waistline” or “dresses with puffed sleeves.”

On another front, the chain has plans to use Amazon Redshift as a data warehouse. "We will reroute data traffic from Amazon S3 to the data warehouse. It is a project that is already ongoing and we want to expand its use even further," reveals Lourenço.