Sophos Teams Up with Aiven and AWS to Drive Down Data Platform Costs by 30%

Sophos, a global provider of advanced security solutions for defeating cyberattacks, needed to address the rising costs of its data platform, which processes 50 terabytes of data daily and continues to grow. The company implemented a Bring Your Own Cloud (BYOC) solution from AWS Partner Aiven. The solution enables Sophos to deploy Aiven services directly to its AWS account and control compute, storage, and networking services. Using the Aiven BYOC platform, Sophos cut its data platform operating costs by 30%, focuses more on higher value tasks, enables a mean time to incident detection of 5 minutes, and scales on demand to support 5% monthly data cluster growth. 

Sophos is on a mission to develop products and services that provide the world's most effective cybersecurity solutions. The company delivers products powered by threat intelligence, AI, and machine learning to secure users, networks, and endpoints against ransomware, malware, exploits, phishing, and a wide range of other cyberattacks. Sophos Central, the company’s integrated cloud-based management console that runs on Amazon Web Services (AWS), is the center of a cybersecurity ecosystem that includes a centralized data lake with open APIs available to customers, partners, developers, and other cybersecurity vendors.

Sophos relies on Apache Kafka for data stream processing and data pipeline management. “We use Kafka to ensure the delivery and resiliency of our mission-critical data because data loss is unacceptable for our business,” says Brian Campbell, senior director of software engineering at Sophos. The company’s Kafka data pipeline processes 50 terabytes of data daily, a number that continues to grow. This data growth has led to increased operational costs. “Our data platform is expensive,” says Campbell. “We are constantly trying to tune the platform and find savings where possible.”

In addition, Sophos IT staff spent too much time manually managing and maintaining the Sophos Central infrastructure. For example, IT employees had to manually manage clusters to ensure reliability and monitor each cluster. If there were problems, staff members also had to provide technical support as necessary. To address these challenges, Sophos wanted to automate infrastructure management so IT could focus more on supporting new initiatives at the company.

To meet its business goals, Sophos began working with Aiven, an AWS Partner that offers a platform for optimizing data and AI workloads in the cloud. “We liked the fact that Aiven is an AWS Partner and also provides a fully managed streaming platform for Kafka,” says Campbell. Sophos was already running Sophos Central on AWS solutions such as Amazon Elastic Compute Cloud (Amazon EC2), Amazon Elastic Container Service (Amazon ECS), and AWS Fargate, a serverless compute engine. Sophos maintains high availability and reliability for Sophos Central across nine AWS Regions globally. “AWS allows companies like ours to focus on our SaaS service rather than a data center,” Campbell says. “I don’t want to spend my time plugging in cables in a warehouse.”

Sophos chose to implement Aiven’s Bring Your Own Cloud (BYOC) solution, which Sophos can use to deploy Aiven services directly to its own AWS account. The BYOC solution only requires specific network access to deploy and manage open source data service, allowing Sophos to customize its network. Taking advantage of the Aiven BYOC, Sophos uses an Amazon Virtual Private Cloud (Amazon VPC) for Aiven services, created within a specific cloud region. Sophos accesses this VPC through an Internet gateway. As a result, Sophos controls all compute, storage, and networking services and the associated costs.

After conducting a successful proof of concept with Aiven, Sophos moved more than 50 data pipeline clusters to the Aiven BYOC, spread across nine AWS Regions. Through best practices and automation, Aiven helped Sophos complete the migration to AWS services three months faster than expected. Sophos also completed the migration with zero downtime, which was essential to ensure the protection of mission-critical data for thousands of customers. “If there had been a production incident during the migration, it could have led to us not protecting our customers,” says Campbell. “If we’re missing detections—or missing data that might be used to research other detections and threats—then we’re not protecting our customers. Fortunately, by working with Aiven, we avoided that.”

By migrating Sophos Central to the Aiven BYOC, Sophos reduced its operational costs because it could deploy Aiven services directly to its AWS account. “We have seen a cost savings of more than 30 percent by implementing Aiven BYOC on AWS,” says Campbell. “We are constantly working to manage our operating expenses because it gives us a competitive advantage by bringing down the overall unit costs on the products we sell to customers. Cost savings is critical to our business, and Aiven is playing a big part in this.”

Sophos has also reduced the time previously spent on managing and maintaining the Sophos Central infrastructure by moving to the Aiven BYOC. “Aiven manages versioning and upgrades, and ensures our clusters are reliable while providing monitoring for those clusters and support if there are issues,” Campbell says. “That frees up my team to do higher value work because we don’t have to spend time managing clusters or doing upgrades.” Sophos is also meeting its latency requirements by using the Aiven BYOC. “Our requirement for mean time to incident detection—from data processing to an analyst or customer being able to see a problem—is 5 minutes. And we’re meeting that requirement with the Aiven solution,” Campbell says. Sophos is also considering working with Aiven to adopt additional technologies in the future. “Our data clusters are growing, with 5 percent more data coming in each month,” says Campbell. “We will support that growth with Aiven and AWS.”

Sophos, headquartered in Oxford, United Kingdom, provides advanced security solutions for defeating cyberattacks, including Managed Detection and Response (MDR); incident response services; and a portfolio of endpoint, network, email, and cloud security technologies. Sophos defends more than 600,000 organizations and more than 100 million users worldwide from active adversaries, ransomware, phishing, malware, and more