Founded at the end of 2013 and now with 60 employees, Ividata is a Paris-based startup that specializes in big data. Through its emphasis on innovation and with extensive investment in research and development, the firm brought to market a retail insights product called
A step up from normal data-gathering services, Ivizone empowers businesses by using this customer data to increase brand awareness and loyalty. In the words of Sidney Burks, chief technology officer at Ividata, “Our aim is to take big data, give it a real purpose, and generate return on investment for our customers.”
At first, Ividata used a number of platform-as-a-service (PaaS) providers to power its infrastructure. This included Engine Yard for web service hosting, MongoLabs for data persistence, and IronIO for data processing and ingestion. Using these PaaS providers, the company was able to get its operations up and running, and get
Very soon, however, Ividata encountered spiraling costs and data processing barriers. As a firm with a keen focus on innovation, it found that this was putting limits on its fast-expanding business. “Our revenue in 2014 was around $3 million, and we aim for that to more than double to $7 million in 2015,” says Burks. “To grow at this rate, we need to have responsive, scalable IT that enables us to deliver a truly innovative service to our customers, and do so reliably.”
In looking for alternative solutions, Ividata evaluated various offerings on the market before choosing Amazon Web Services (AWS). “In terms of transparency of offering, flexibility of service, and maturity of platform, we could see that AWS was the right fit for us,” says Burks.
Given the issues with its previous infrastructure, scale and cost-effectiveness were central to Ividata’s decision. “We knew that scalability and cost
Implementation began in September 2014, and, just two months later, the firm’s infrastructure was running entirely on AWS.
Raw location data from WiFi-enabled smartphones—about 1 million data points a day—reach the API server, which is hosted on AWS Elastic Beanstalk, using Amazon Elastic Load Balancing to route traffic between Amazon Elastic Compute Cloud (EC2) instances across multiple Availability Zones (AZs). “This means that if there’s a problem with one data center,” says Burks, “or a server crashes, our systems stay online and we maintain high availability.”
The logs are then written to Amazon Kinesis servers. There, Amazon Kinesis Client Library (KCL) and AWS Kinesis Connector Library aggregate and compress the data to a ratio of 90 percent. The data is stored in Amazon Simple Storage Service (Amazon S3) and analyzed in real time via Amazon Elastic MapReduce (Amazon EMR), before using spot instances to store the resulting analysis in the Amazon Relational Database Service (Amazon RDS). The web-based metrics dashboard that’s used by customers is also hosted on Elastic Beanstalk—again across multiple AZs for maximum availability. Burks says, “Our web applications and RDS databases have had 100 percent uptime since migrating to Elastic Beanstalk and implementing the Multi-AZ configuration.”
The firm uses Amazon CloudFront to speed up asset caching. Across its infrastructure, the business also uses Amazon Route 53, Amazon DynamoDB, and Amazon Glacier.
AWS security features were another factor in choosing AWS. “Because we deal with potentially sensitive data, security was a major requirement for us,” says Burks. “We’re currently using AWS Identity and Access Management to manage security on the data access layer. We’ll also be migrating to Amazon Virtual Private Cloud to help our compliance requirements.”
In the future, Ividata plans to integrate Amazon Simple Notification Service (SNS) and AWS Lambda to further optimize its application.
For all startups operating in fast-moving markets, cost-effectiveness in IT is essential. Ividata is no different in this respect. Since its migration to AWS, the company has experienced a boost to operational agility and service innovation at a significant cost savings. Burks says, “Before, our bill for storing one terabyte of location data came to $5,000 a month. With Amazon S3, on which we were able to apply a 90 percent compression ratio, we’re looking at just $5 a month. That’s 1,000 times more cost-effective than our previous solution, which makes a huge difference to our operating expenditure.”
The pricing benefits of AWS don’t stop at data storage. “Running our web servers is now around four times more cost-effective than on our PaaS solution, having decreased from $125 to $30 a month on AWS Elastic Beanstalk. Together with our data storage savings, this has had a significant and immediate impact on our flexibility because our costs are so much lower than they were before,” says Burks. Plus, the speed of data processing on AWS compared with the previous PaaS solution is proving a big advantage. Burks continues, “Our nightly data processing job used to take 12 hours, but with Apache Spark on Amazon EMR, it now takes just 30 minutes. This allows us to iterate much more quickly in optimizing our algorithms.”
AWS has also given Ividata the scalability it requires as it pursues new business. This is having a direct effect on the firm’s competitiveness in a market that relies on service innovation to retain and attract customers. Burks says, “We can handle much larger data volumes, and we can also scale out in a much more reliable manner. As a direct result, we can now offer parts of our service as a free tier for customers to try, so they can see the benefits without any contractual engagement upfront. We just didn’t have the capacity or flexibility to do this before.”
Burks also highlights the advantages of increased automation. “Using numerous Amazon SDKs, we have much more power to automate every aspect of our infrastructure. And with Auto Scaling groups and multi-AZ features, we have confidence that our servers will stay online. Our deployment processes are completely automated and we don’t have to invest in training each member of the team on different deployment practices. This is helping us further lower our operating expenditure.”
Looking to the future, Ividata has a number of bright prospects with some famous names in fashion retail, as well as a major multinational bank. “With the AWS service behind
In addition, Burks has found that the company’s relationship with AWS is helping it maintain its focus on innovation. Through AWS Support at the Business level, and with direct access to a dedicated account manager, not only has Ividata benefitted from extremely fast responses to technical issues, but it also has the support to test new features and drive its own product. Burks concludes, “The level of contact with AWS has been impressive. It’s been one of the highlights of working with AWS. We feel like we’re part of its roadmap, and as we expand our use of AWS we’ll be able to adapt our solution to market needs, releasing new products to market much faster and boosting our competitive advantage.”
To learn more about how AWS can help with big data analytics, visit our Big Data details page: http://aws.amazon.com/big-data/.