Curalate makes social sell with AI using Apache MXNet on AWS

Curalate helps brands convert social influence into sales. The Philadelphia-based startup makes it easy for digital-savvy consumers to make associations between products they see on social channels and where they can buy the very same products. Previously, consumers had to hunt through online catalogs on their own to find a specific product from a social image. With Curalate, products can be linked automatically to where they can be bought.

And since every social post is a potential influence, brands can use the words, images, and memes generated by social users to create a choir effect with lots of voices.

“We are bringing the discovery process online. The same way you might see something when you walk through the store, our products help you make that connection, say on Instagram or other social channels,” explains Jesse Brizzi, Research Engineer at Curalate.

Over 1,000 brands and retailers, including Neiman Marcus and J.Crew, and over 100 million consumers a month use Curalate. These brands realize 20-to-50 percent increases in revenue per visit using Curalate’s social commerce platform and up to 60 times increase in products discovered. For Karl Lagerfeld, tagging photos with Curalate’s image recognition technology allows the fastest and most effective way for its customers to shop for their perfect pieces.

Driving Curalate’s success is deep learning technology on AWS, which helps to make products more discoverable on social media. Curalate’s intelligent product tagging (IPT) service allows customers to automate the collection and curation of user-generated social content in the pipeline, and then connects it to the right product in the catalog. IPT is trained using deep learning algorithms, developed using the Apache MXNet framework, to make it simple for brands to match images with products to recommend to shoppers.

Brizzi notes that, “initially clients had to manually tag the products. Where they would click on the item, tag it, and then go inside their product catalog and manually make the match. Now with deep learning we can make the match and they just have to confirm it. With IPT they are saving time, improving accuracy, and helping consumers find things easily without wasting time.”

Using MXNet, Curalate can quickly build and deploy production models on AWS—training the models on Amazon EC2 P3 instances, pulling data from Amazon S3 and drawing on API services—a steep challenge given the millions and millions of images that need to be ingested and analyzed. MXNet and Gluon, a high-level API interface, allowed Curalate to streamline their end-to-end model development without sacrificing training speed.

“Speed was a key reason we chose MXNet over TensorFlow. We care about moving fast and moving lean from prototyping to production, and MXNet helps us do that. We were looking to make our lives a lot simpler especially in the transition from experimentation to production,” says Brizzi.

MXNet’s support for Scala APIs was also a factor in considering the deep learning framework. “We had to deal with a lot of native code. We used to have to write our own bindings to go from C++ to Scala. This was a pain to manage and keep updated. Using MXNet and Gluon made it way easier to write experiment code and iterate in the Python language and then transfer the models from Python to Scala,” says Brizzi.

For Curalate, the next step is to scale up its platform. As the popularity and influence of social media continues to grow, there will be even more product images to ingest and analyze and connect.

Learn more about how Curalate uses MXNet in this technical deep dive. To get started with deep learning you can try MXNet as a fully-managed experience in the Amazon SageMaker machine learning platform, or use MXNet with the AWS Deep Learning AMIs (Amazon Machine Images), which provide preconfigured development environments for deep learning.

About the author

Cynthya Peranandam is a Principal Marketing Manager for AWS artificial intelligence solutions, helping customers use deep learning to provide business value. In her spare time she likes to run and listen to music.