Distributed Media Lab’s Chief of Technology on Why He Wanted a Serverless Architecture

Sam Parnell has been an AWS evangelist most of his career. While Chief Technology Officer at popular sports news site Bleacher Report, he ran the company “100 percent on AWS.” When Turner Broadcasting acquired Bleacher Report in 2012, Parnell got involved with aspects of the media giant’s infrastructure and again started moving critical operations into the cloud.

“Bleacher Report was a really good case study for the rest of the organization,” he says. “To be able to say: this really works, and you can do it for less money and with fewer people.” They were the model students.

Now he’s chief of technology at Distributed Media Lab, a Palo Alto-based media technology startup. Early on, the company tried Google Cloud. But according to Parnell, the performance wasn’t cutting it and the security features weren’t up to par. And to position Distributed Media Lab for high-paced growth, Parnell wanted a completely serverless architecture. AWS had the more mature offering, he says.

“Everything we’ve built runs through AWS Lambda,” Parnell says. In fact, in the first year of Distributed Media Lab’s operations, they’ve never spun up an EC2 instance. They’ve taken all their underlying processes and content and created “an event-based micro-services architecture,” he says. “That allows an engineering team to scale efficiently.”

To understand this decision, it’s first important to appreciate Distributed Media Lab’s niche in media. Publishing on the web is challenging. Everyone is trying to maintain control of their content. Writing, photos, and videos are constantly ripped off by less reputable actors. But there is also the question of when an outlet intends to share their content with another. If site A wants to feature site B’s content, should they email it, give each other access to their respective content management systems, or try something else entirely? Given how time-consuming it can be to manage a CMS, good content often gets lost.

“Content syndication has been really broken,” Parnell says. To get more eyeballs on their work, “Publishers need more audience development options than just Google and Facebook.”

Parnell thought a lot about this problem while he was at Bleacher Report and Turner, as did his CEO and co-founder David Gehring while Gehring was at Google and YouTube. Distributed Media Lab ended up harnessing accelerated mobile pages (AMP), a technical standard for fast loading mobile web pages, and creating embeddable code publishers can drop on their own sites to share partner content. It’s not just an html iframe tag; the overlay comes with its own analytics and paywall capabilities.

“That allowed us to leverage a content distribution standard that encapsulated the publisher’s business model,” Parnell says.

One of Distributed Media Lab’s earlier customers was the nonpartisan journalism center CALmatters. In the run-up to the 2018 U.S. midterm elections, the group a built a helpful guide for California voters about candidates, ballot initiatives, key issues, and relevant news—but had nowhere to share it. So Distributed Media Lab enabled newspapers like the Long Beach Post to embed the entire guide on their homepage, flowing with the rest of the page’s brand and design. If a user clicked on a CALmatters article, they remained on LBP’s website, as though the CALmatters guide were another section of the site. Yet the content and ads in the viewer remained tied to the original publisher’s servers.

“Then everyone has the opportunity to make money from the content,” Parnell says. He notes the technical process is far easier than having engineering teams wrangle RSS feeds and custom processes to ensure the HTML and CSS stay intact. “Forget all of that,” he says. “Just take two lines of code and you can syndicate content across a network of publishers.”

Distributed Media Lab’s only been around since mid-2017 but the company continues to add publishers each month. Parnell is already thinking about how to structure his engineering teams for the future. That is why he chose serverless architecture. The idea is to keep the code for each process and sub-process separate, in containerized blocks of code.

“All of the different pieces that go into the UI, administrators, publishers—they’re all broken down into components that can be updated and deployed separately. That keeps teams from trampling on each other,” he says. “There’s no big server everyone works on at the same time.” He says that enables efficient scaling. There’s less coordination when you’re not all working on the same code base, he says.

Everything is done through AWS. They use AWS Lambda to run the backend services, which are built in Node. They use SQS to manage the event messaging between the services. Distributed Media Lab uses Amazon API Gateway to power the customer tools and management interfaces, and AWS Cognito to manage the logins and access.

But all in all Distributed Media Lab is still young—or mean and lean, as Parnell puts it. “We’ve managed to keep the design of the system reasonably concise,” he says. “I’d be a huge advocate of others trying this.”