umlaut Uses Amazon S3 to Cut Storage Costs by 30%, Speed Business Growth

Based in Germany, umlaut offers end-to-end advisory and fulfillment services to customers worldwide. Its 4,300 consultants and engineers span a collective of 20 consultancies and engineering firms across 50 global locations. In the telecommunications sector, umlaut provides an array of independent technical and management consulting services and also specializes in collecting cell phone data to analyze cellular network quality. “We provide connectivity testing, so telecommunications companies and other customers can identify problems with connectivity, connection speed, or voice quality problems in certain areas,” says Hakan Ekmen, global chief executive officer telecommunication at umlaut. “To do this, we collect and process data from millions of customers every month.”

Increasingly, umlaut needed to respond to customer demand for faster access to multiple terabytes of data, which was challenging because the company relied on an aging on-premises infrastructure that was not sufficiently scalable. “In our data center, we had a local six-node server cluster that wasn’t able to grow with our needs,” says Ekmen. “As a result, it took us nearly two weeks to set up new nodes to increase the cluster space and capacity. Especially for our data ingestion pipeline, we wanted to ensure we could manage storage so there would be no bottleneck with data processing. We knew we had to move to the cloud to solve the problem.”

To meet its scalability requirements, umlaut chose to move its application environment to Amazon Web Services (AWS), using AWS Snowball to move data to Amazon Simple Storage Service (Amazon S3). The company chose AWS Snowball to transfer 38 tebibytes (TiB) of backup data to the cloud so it could make calculations using the same dataset locally—and in the cloud—while migrating to AWS. “Our experience using AWS Snowball was great. We simply connected it to our network and used the specific client to upload the data,” says Ekmen.

umlaut now uses Amazon EMR to process petabytes of cellular device data, sending it to an Amazon Redshift data warehouse for processing product-optimized datasets and ultimately into an Amazon S3 data lake. “We primarily store connectivity test data in the Amazon S3 data lake. The data shows if users can easily go online at a specific location,” Ekmen says. umlaut enhances security for its data lake by using Amazon S3 Block Public Access to restrict access to data. Employees rely on AWS Glue to prepare data for analyses through automatic extract, transform, and load (ETL) processes and then use Amazon Athena to analyze the data before preparing it for customers. umlaut also uses Amazon S3 Glacier for long-term storage of test data. “After we process a lot of the data, we move it to Amazon S3 Glacier because we typically don’t need to touch it anymore,” Ekmen says.

As umlaut’s business grew, its storage size grew as well. “To continue to optimize costs, we sought a low-cost, secure, and reliable storage option for our long-term data while keeping our frequently accessed data on Amazon S3 Standard,” says Ekmen. The company chose to further optimize its storage environment by moving data that it didn’t need to access for long periods to Amazon S3 Glacier Deep Archive storage class, which provides highly secure, durable object storage for long-term retention across multiple Availability Zones.

umlaut uses Amazon S3 Storage Class Analysis to understand data access patterns, and it optimizes costs by using S3 Lifecycle Policies to transition data from Amazon S3 to S3 Glacier Deep Archive. “We knew we could save a lot of money by using Amazon S3 Glacier Deep Archive, and it offers the same level of durability as Amazon S3,” says Ekmen. “When we have a customer request for data that is older than one year, we usually have at least a few days to respond. We were able to optimize costs based on our business needs; this was the perfect solution.”

umlaut captures cellular data in the Apache Avro data serialization framework, which uses the JSON programming language for defining data types. The company receives the data and uses Amazon Kinesis Data Firehose to stream it to its Amazon S3 data lake, and then uses an Amazon S3 Lifecycle Policy to migrate that data to S3 Glacier Deep Archive. “We process 3 terabytes of data daily, and we transition that data to S3 Glacier Deep Archive after seven days. We currently have a petabyte of data in S3 Glacier Deep Archive, and it’s growing all the time,” says Ekmen.

Moving from Amazon S3 Glacier to S3 Glacier Deep Archive, umlaut can store cellular connectivity data at a lower cost. “We’re reducing our monthly storage costs by 30 percent using Amazon S3 Glacier Deep Archive for long-term data storage and retention, while using Amazon S3 for the initial data analysis that requires faster data access to process data faster,” says Ekmen.

umlaut is now able to innovate faster and reduce the time to bring products to market, because its IT teams can now focus on developer applications and not on managing storage infrastructure. “We are reinvesting the money we are saving with S3 Glacier Deep Archive back into our business, to innovate, grow, and also enhance our application to give our customers a better experience,” says Ekmen.

umlaut can now process and store data from mobile devices more quickly and deliver data to customers several days faster than before. “Since moving to Amazon S3, we have grown almost 6,000 percent, from 100,000 users to several million users, because we can process and store multiple terabytes of data every day,” says Ekmen. In addition, umlaut can capture more data types, providing the ability to serve more customer use cases. For example, by increasing the number of KPIs captured from phones, umlaut was able to provide more valuable data for customers and create new products. 

umlaut is increasing its analytical capabilities to better serve customers in the automotive, aerospace, energy, and telecommunications industries. “By taking advantage of Amazon Athena and AWS Glue in our Amazon S3 data lake, all our employees can easily run queries on connectivity data,” says Ekmen. “As a result, we can do different analyses for customers in different industries. For example, we can create heat maps to help energy customers identify gaps in their mobile networks.”

umlaut plans to expand its Amazon S3 data lake to analyze even more data. “We expect to further improve our data analytics capabilities as we process more and more data on Amazon S3,” says Ekmen. “Amazon S3 gives us the scalability and storage cost optimization we need to continue growing our business, because of the various storage classes and pay-as-you-go model.”

To learn more, visit aws.amazon.com/s3.