Driving Business with Advanced Monitoring and Logging Using Amazon OpenSearch Service with Web & Mobile Team at Hapag-Lloyd AG

The Web & Mobile department of German shipping company Hapag-Lloyd, a distributed team based in both Hamburg and Gdansk, is responsible for developing the customer channel’s web and mobile products, providing online applications for customers to get quotes, book containers, and track their shipped containers. “We are a business-to-business company with 300,000 customers in our databases, many of whom use our online offerings daily,” says Björn Klatt, director of the Web & Mobile team at Hapag-Lloyd in Hamburg. To better serve these customers, the Web & Mobile team migrated its solutions to AWS in 2020. On AWS, it modernized its stack to have a serverless architecture using AWS Lambda, a compute service that runs code in response to events and automatically manages the compute resources. “Using AWS services and capabilities in our department, it was possible to scale up solutions that are safe and customer-oriented,” says Borys Wirkus, director of the Web & Mobile team at Hapag-Lloyd in Gdansk. “We, in Web & Mobile, are AWS frontrunners in the company, because this is what our customers are expecting: fast, reliable, and suitable solutions.”

Following the migration, the 70-person Web & Mobile team had over 1,000 AWS Lambda functions deployed across multiple accounts, which posed challenges for developers. With eight product teams working together as one, the Web & Mobile department is constantly developing new features and new products on its technology stack. The team needed clear monitoring and dashboards to make it simpler to track down and resolve technical issues and to use data to make better business decisions. In response, Hapag-Lloyd Web & Mobile created a centralized logging solution using Amazon OpenSearch Service. “We created centralized logging so that software engineers and business stakeholders could have one place to search for technical and business metrics,” says Grzegorz Kaczor, cloud architect in Web & Mobile at Hapag-Lloyd.

The centralized logging solution that the Web & Mobile team created is a comprehensive system that collects log data from a variety of components, including AWS Lambda functions and Amazon API Gateway, which is used to create, maintain, and secure APIs at any scale. The team also uses asynchronous services like Amazon Simple Notification Service (Amazon SNS), a fully managed Pub/Sub service for A2A and A2P messaging, and Amazon Managed Streaming for Apache Kafka (Amazon MSK), used to securely stream data with a fully managed, highly available Apache Kafka service. “The system automatically produces technical log events and is complemented by a style guide for integrating product-specific events, tailored to their unique characteristics,” says Marcin Szałomski, software architect in Web & Mobile at Hapag-Lloyd. “Using this rich log data, we have engineered a solution that not only supports our day-to-day operations but also automates certain tasks.” After the log data is processed, partitioned, and transformed, it is ingested into the Amazon OpenSearch Service.

Using the centralized logging system, the Web & Mobile team can instantly monitor its applications’ status, reducing reliance on customer reports to identify service issues. Upon deploying new releases, the team can visually confirm that the application is running properly right away.

Additionally, Amazon OpenSearch works seamlessly with a notification solution, so the team can automatically query the data for additional context whenever an outage launches an alarm.

When issues do arise, developers can get to the root cause more quickly and resolve them faster.

The centralized logging solution is also a source of data for making business decisions for Hapag-Lloyd Web & Mobile. “Technical logs, when analyzed, give us business context and information, too,” says Dawid Kuziemski, product manager of Web & Mobile at Hapag-Lloyd. For one Hapag-Lloyd product, Quick Quotes—which customers use to generate shipping container quotes in 30 seconds—the Web & Mobile team used Amazon OpenSearch Service dashboards to determine the importance of certain features for customers. In 2022, a relevant portion of Hapag-Lloyd’s business was booked through Quick Quotes. However, there was a functionality barely used by its customers that was difficult to integrate into a new release. Based on the data collected, the product managers decided not to include it in the upgrade, which saved the developers time and reduced the overall technical debt of the solution. The team also used data to determine which default value should appear in one Quick Quotes field and revealed another field where it needed to standardize the possible options. “We use the Amazon OpenSearch Service dashboard to make data-driven decisions, which is very important to us,” says Felipe Barrera, product owner of quotation tools in Web & Mobile at Hapag-Lloyd.

“We have much better flexibility as a team because—using Amazon OpenSearch Service and the centralized logging—we have more information and we can measure success,” says Kuziemski. “We can learn faster and iterate faster on our software because we can measure if we are improving.”

After successfully establishing a centralized logging and monitoring solution, the Web & Mobile team at Hapag-Lloyd is eager to further its innovation using AWS. “The digitalization of the container industry is an ongoing journey—we are dedicated to making strategic, purposeful decisions so that we can satisfy our customers’ needs,” says Wirkus. “With the rapid expansion in our sector, having secure and responsive solutions that address customer issues promptly is essential.”

The Web & Mobile team is pleased with the level of support it has received from AWS. “It’s great that we work alongside an AWS team with experienced solutions architects,” says Kaczor. “They’re very customer-oriented and always ready to provide support and guidance on architectural decisions.”