Resilient video encoding across multiple AWS regions

The need for resilient technology workloads transcends all industries. It is particularly important for the media and entertainment (M&E) industry because disruption of media workloads may lead to customer churn or brand tarnish. A misunderstanding of resilience as a concept can lead to workloads that may not have resilient architecture or workloads that are over-engineered.

The AWS Well-Architected Framework defines resilience as “the capability to recover when stressed by load (more requests for service), attacks (either accidental through a bug, or deliberate through intention), and failure of any the workload component.”

AWS Elemental MediaConvert is a service from Amazon Web Services (AWS) that performs video encoding in the cloud.

Considerations before getting started

AWS Regions are built with multiple isolated and physically separate Availability Zones (AZs). This approach lets you create highly available Well-Architected workloads that span AZs to achieve greater fault tolerance. This satisfies the availability goals for most applications, but there are some general reasons to expand beyond a single region:

• A need to reduce latency for different parts of the world as an application’s user base grows geographically dispersed.
• Reducing Recovery Point Objectives (RPO) and Recovery Time Objectives (RTO) as part of a multi-Region disaster recovery (DR) plan.
• Local laws and regulations may have data residency and privacy requirements.

In this blog post, we discuss how to deploy architecture for a video-on-demand encoding workflow across two regions. In the very rare event of region failure, the workflow will continue to ingest and process video files even if one of the regions is impaired.

The architecture

Figure 1. Multi-region VOD design architecture

For a single point of video file ingest into any region, we use Amazon S3 Multi-Region Access Points to deliver built-in network resilience, built on top of AWS Global Accelerator to route Amazon S3 requests over the AWS global network. By dynamically routing your requests to the lowest latency region, Amazon S3 Multi-Region Access Points increases upload and download performance by up to 60%.

Once the source input video file is uploaded via Amazon S3, Multi Region Access Points to the closest Amazon S3 Bucket in relation to the user PUT request. When a file is written to the source Amazon S3 bucket, it triggers an AWS Lambda to submit the AWS Elemental MediaConvert transcoding job. After the transcoding job is complete, the output files are available in another Amazon S3 bucket.

The same file encoding workflow is replicated in another region (in this example EU-West-1)

The destination Amazon S3 buckets in both regions will have Cross Region Replication enabled. If one of the regions fails, encoded assets can still stream via Amazon CloudFront to end user devices.

This video encoding workflow is a simplified version of the AWS VOD Foundation Solution with only the necessary services for the purposes of this example.

Cost disclaimer

The Free Tier (https://aws.amazon.com/free/) does not cover the MediaConvert and S3 Multi-Region Access Point costs to build this workflow, so the file ingest and processing will incur charges. You are responsible for the cost of the AWS services used while running this workflow.

For a step-by step guide on how to implement a Multi-Region VOD workflow, please visit the File Streaming with AWS Media Services workshop.

Cleaning up

To avoid incurring future charges, delete the resources created in this blog post, including S3 Multi-Region Access Points, CloudFront Distribution, S3 Buckets, and Lambda Functions.

Conclusion

In this post, we discussed how to deploy an automated, file-based video encoding workflow across two regions using Amazon S3 Multi-Region Access Points, AWS Lambda and AWS Elemental MediaConvert. Amazon CloudFront is used as the CDN to cache the content near the edge and provide origin failover in case of region failure.

Learn more

If you would like to explore more video-on-demand workflows built on AWS, visit https://aws.amazon.com/media/supply-chain-archive/. If you are looking to migrate your VOD library to AWS, see https://aws.amazon.com/blogs/media/top-10-things-to-consider-when-migrating-your-vod-library-to-aws/