News Distribution Network (NDN) provides profitable, next-generation media solutions for leading publishers and brands, enabling them to quickly and easily give users access to compelling video content. Founded in 2007, NDN set out to provide free solutions for online publishers and content creators who were struggling to navigate the online monetization ecosystem. In 2009, NDN introduced the Digital Media Exchange, a comprehensive ground-up solution for publishers, providing fully monetized access to sophisticated video players and premium brand-safe content. In 2010, the Associated Press (AP) selected NDN as its syndication partner for AP's 4,500 member and associate member sites; the following year, NDN added natural language processing and semantic analysis to its portfolio of services, allowing NDN to automatically deliver the most continually relevant video to news stories.
Today, NDN is the fourth largest online video property in the US (comScore, January 2014), driving 600 million video views per month across 146 million unique viewers. NDN is shaping the future of content distribution and website monetization with an integrated multimedia solution that helps partners drive user engagement and generate revenue.
In the publishing business, costs are critical. The last 20 years have seen the rise of the Internet and surges in mobile technology, providing robust new channels for content of all kinds—but increasing competition and changes in technology have also created financial challenges, particularly for publishers of news. In 2007, when NDN was founded, publishers and content creators alike were struggling with video: it was tremendously popular, but hard for publishers to fund and for creators to monetize. Publishers had to pay for content, licensing, and streaming, not to mention investing in staff to discover, curate, and sell the content. Content creators didn’t have reliable channels to sell videos.
NDN saw the opportunity to fill a gap in the market, and stepped in to create a new model for publishing video content. “If we could solve the challenge of distribution and reach for providers, and help publishers get content at the same time, we’d be in the sweet spot,” says Eric Orme, NDN Chief Operating Officer and Chief Technology Officer. “We decided to provide publishers with a free Software as a Service (SaaS) solution.” By creating a digital media exchange on a revenue-sharing model, publishers and content providers can share the ad-revenue profit that each video earns. See below for an illustration of the NDN Digital Media Exchange.
Figure 1. NDN Digital Media Exchange
That model and its underlying technology would be disruptive to an industry that was still struggling to define a clear, profitable way to publish video. “Publishers were making complicated content deals, and it was very difficult to track usage, data, and revenue across a multitude of channels,” says Bobby Bond, NDN Vice President of Engineering. “We knew that to truly make a difference, we had to take cost out of the equation.” NDN would need to scale up quickly to meet the demand for video, without having to spend the time or the resources to build a traditional on-premise infrastructure. “We set out to build a profitable business early, and that meant we needed to be smart about how we spent our resources,” Orme says.
The company did some prototyping and investigated different cloud options to make sure it would be able to scale with demand, without breaking the bank.
After researching its options, NDN chose Amazon Web Services (AWS) as its cloud provider. “AWS gave us the ability to predict what our usage would cost and let us move those costs to the margin, without having to worry about capital expenditure,” Orme says. “As it was, we grew so fast that a data center would have been prohibitively costly, not to mention time-consuming.”
Using AWS gave NDN multiple services to quickly develop a global media exchange that would provide always-on availability without requiring a big staff to run it. Premium publishers can take advantage of NDN’s media exchange products and tools to discover and choose a tremendous amount of high quality content and instantly monetize it at no cost. Content providers are able to easily distribute their content profitably to these premium publishers and have full transparency throughout the platform.
Amazon Elastic Compute Cloud (Amazon EC2) serves up videos to users, and Elastic Load Balancing helps automatically distribute incoming web traffic across instances. Amazon ElastiCache is used to cache the player services calls, including which thumbnails, videos or playlists to load next. By using AWS, latency has been reduced to an average of 40 milliseconds, with a payload that tells the video player what to load on the user’s page.
Publishers can place videos on pages using traditional embedding or CMS integration, but NDN has developed other methods that decrease the costs and time needed while increasing the efficiency. Instead of disrupting editor work flows or of tasking teams with costly, time-consuming exercises, publishers can simply use NDN’s bookmarklet to embed video on any page they like, regardless of the publishing CMS they use. The bookmarklet gives contextual recommendations for video that matches the page content and allows for great flexibility to accommodate workflows. NDN has also automated the process of embedding timely and relevant video in pages without the need for human intervention, yet still allowing for full editorial overrides and control.
For storage, NDN uses Amazon Simple Storage Service (Amazon S3) as a depository for more than 100,000 new video titles each month. NDN is leveraging the power of Amazon S3 to store player assets, server logs, analytics data and other data that requires a high amount of availability and durability. Server logs are stored in Amazon Glacier; Amazon Elastic MapReduce is used to analyze some of the raw data from Amazon S3.
NDN uses Amazon Redshift to analyze nearly 600 million ad impressions and generate reports. Redshift has given NDN the ability to scale its data warehouse, enabling the company to focus on analyzing the data and running jobs. “With the scale of our data, our solution had to be fast and scalable,” Bond says. “The ability for us to resize our cluster and import massive amounts of data in parallel were key factors in our decision to choose Redshift.” The company also uses Redshift to back up its data. “Redshift also makes backups a snap,” Orme says. “We can spin up a new cluster from a backup and test changes without disrupting the business, which is key for us as a 24/7 operation.”
The company is also using Amazon Kinesis to perform analysis on the massive amounts of streaming data that its users generate, estimating that more than 1 billion calls are made through Kinesis each day. Using Kinesis enables the company to gauge how fast its players are loading in real time. “If we see a player taking more than one second to load, we can send out alerts in real time,” Bond says. NDN collects beacon logs that identify details like which pages contain its videos, how quickly pages load, how many videos users have watched, and at which points during the video did users stop watching. The company also uses Kinesis to track ad net calls and other ways that the video players are being used, as well. Kinesis logs the data to Amazon S3, Amazon Redshift, and Amazon RDS, depending on the type of data it is and what NDN needs to get out of it.
NDN stores customer data and viewer data in Amazon Relational Database Server (Amazon RDS), and uses Amazon CloudFront as its content delivery network (CDN). The company uses Amazon Route 53 for its DNS server, and is looking into using Amazon DynamoDB to store data for faster loading by consuming applications. NDN uses Amazon Elastic Beanstalk to deploy and manage applications. Applications are deployed and launched using Elastic Beanstalk and cutover after UAT has been completed, minimizing downtime for the users.
Using AWS met NDN’s needs for scalable, always-on computing power with predictable costs. By using AWS, NDN can provide its customers with 600 million content plays a month, steadily keeping them in one of the top comScore spots— which in turn increases NDN’s visibility with advertisers. “We have been on top of the comScore list for quite a while; it’s a testament to how fast we’ve taken off,” Orme says. “It’s also helped us to stay privately funded. We’ve been able to be profitable and grow without having to take VC funds.”
The model quickly took off with publishers, enticing news giant Associated Press to partner with NDN in 2010. “We were able to take cost out of the equation,” Orme says. “It’s one of those rare models where everyone gets what they want: content providers can get revenue, and publishers can get premium, brand-safe content that is compelling to their customers. Using AWS has enabled us to build a solid platform that has scaled quickly while becoming a source of profit for our customers.”
Using AWS also allowed NDN to focus on building its business. “We can focus on the problems publishers are struggling with because we don’t have to build out an infrastructure,” Bond says. “Among our competitors, we’re the only one who doesn’t have a destination site. We aggregate and distribute this content in a very quiet way, where it really fits into our customers’ brands.”
The company plans to expand its use of AWS in the coming months. “We are expanding our use of Elastic Beanstalk to handle processing of video transcoding jobs. When news breaks, we must make sure we don’t have a single video stuck in transcoding. We also leverage AWS for notifications with these types of events. Getting the video in and out to publishers quickly is a huge advantage of our platform. We’ve really started to tap into the wide variety of services that AWS offers,” Bond says. “AWS lets us scale and grow—our data footprint has grown massively in just the last 6 months, and with AWS, it’s a snap to manage.”
To learn more about how AWS can help with your big data challenges, visit our Big Data details page: http://aws.amazon.com/big-data/.