If AWS didn’t have Postgres and compatibility with other open-source solutions, we would have had a ton more work to do.
Michael Lorant Senior Systems Engineer, Fairfax Media

The proliferation of digital media has turned publishing on its head.

Fairfax Media is an Australian conglomerate with publications such as The Age and The Sydney Morning Herald, two of the region’s oldest and most widely read newspapers. Fairfax Media recognized the tidal wave of change in its industry and the need to do more to engage readers. It also wanted to modernize workflows and empower its internal audience of more than 1,000 journalists and editors, as well as 15–20 developers. Michael Lorant, senior systems engineer, at Fairfax Media, says, “We were ripe for disruption, and the easiest solution was to disrupt ourselves.” Thus, began a ‘blank slate’ greenfield effort to completely reinvent Fairfax Media’s publishing platform.

The new projects’ architecture was to be built on Amazon Web Services (AWS) using Kubernetes, an open-source container management system that automates application deployment. From a systems perspective, agility and flexibility were top priorities. Lorant’s team felt their technology setup was inhibiting rather than enabling developers. He says that having the agility to experiment and quickly adjust to market demands was key to creating the “Fairfax of the Future.”

Many companies use AWS to run Kubernetes, but very few had attempted to use it for enterprise-scale workloads. This proved a major challenge, but by working closely with AWS and embracing the open-source concept, Lorant’s team was able to customize applications. “Nearly every piece of this solution is open source,” Lorant says. The new CMS is powered by Postgres relational databases, which Fairfax manages using Amazon RDS for PostgreSQL. “If AWS didn’t have Postgres and compatibility with other open-source solutions such as Snowplow, we would have had a ton more work to do.”

In addition to Postgres relational databases, Fairfax uses Amazon DynamoDB for secrets storage using Confidant, an open-source secrets manager that is tied to Amazon Key Management Service (KMS) for data encryption. Fairfax relies on Amazon Kinesis to collect and process data as part of its analytics pipeline. It also counts on Amazon Elastic Container Registry (Amazon ECR) to store and manage container images. The enterprise also uses Amazon Simple Storage Service (Amazon S3) for its long-term storage needs, as well as several other managed services. “Our philosophy is that if AWS has a service, we will use that first,” Lorant explains. “We are a media organization and not a tech company, so we don’t put energy into things AWS is better at.”

One of the greatest advantages of running Kubernetes containers is a more effective utilization of CPU memory. Fairfax uses 80 Amazon Elastic Compute Cloud (Amazon EC2) instances for its Kubernetes nodes; enterprises that build applications on bare Amazon EC2 instances without Kubernetes can easily run hundreds of instances at any given time. “Most companies get about 10 to 20 percent CPU utilization because they must allow headroom for peak times and segregation of services,” explains Lorant. “In a containerized world, we are able to stack workloads to 70 or 80 percent quite easily, so effectively we are able to use 5 times less computing power, which also means 5 times in cost reductions.”

In December 2017 and February 2018, Fairfax migrated its flagship properties—The Age and The Sydney Morning Herald—onto the new publishing platform. To ensure that the process went smoothly, the enterprise used AWS Infrastructure Event Management, working side-by-side with the AWS team throughout the migration to troubleshoot any issues.

The new publishing platform has revolutionized the way journalists and editors interact. Rather than waiting for emails with hefty attachments, they can follow the progress of a story on Slack, which is updated in real time. There has also been a significant increase in reader engagement. The analytics capability of the new system allows Fairfax to measure all factors concerning user engagement with its content, in near-real time. This includes dwell time on articles and average scroll depth, showing growing audience engagement on key measures. Lorant attributes this to a better understanding of each publication’s audience. “Journalists and editors can create different types of articles based on whom they are trying to market to.”

Fairfax’s digital transformation has fundamentally shifted developer workflows as well. In their new DevOps environment, developers are empowered to build infrastructure on the platform directly.

“The amount of experimentation that is going on is incredible,” says Lorant. “Developers are more inspired, and I think things have become more efficient as well.” Because the new system allows developers to easily reverse operations, innovation is less risky. “Before, more care was taken because shifting back was going to be a long and difficult process,” he says. “The cost of failure is now very low.” Deployment cycles have dropped from three weeks to multiple times per day, so engineers can quickly roll out bug fixes and new features. Every week, Lorant sees a handful of new microservices pop up within the platform.

Readers have also benefited from shorter load times on the site, with no outages reported. Other divisions of Fairfax have taken note of the project’s success. Domain, a real estate marketplace in the Fairfax portfolio, is now looking to adopt Kubernetes for its operations.

Lorant feels Fairfax Media is in a good place and is looking ahead. “We are now examining how to take this forward, but it’s a good position to be in—to know that you have moved to the front of the market and the technology space.”

He concludes, “This project probably would not have been possible without AWS. We had a design that was changing very rapidly and we were building quickly, so we couldn’t be limited in terms of what we were doing. This would not have been possible without a stable cloud infrastructure. That’s what AWS gives you: flexibility and stability.”

Learn more about Kubernetes on AWS