Slack elevates media pipeline with AWS Elemental MediaConvert and Amazon Transcribe

Transforming the way people work, intelligent productivity platform Slack was built with transformation at the forefront. The product team at Slack constantly evaluates areas for improvement based on customer usage and the desire to launch distinguishing features. One such feature, clips, allows users to record and send audio and video files directly with other people, either by uploading media files or recording directly in Slack. To ensure files are high quality and load quickly, Slack leverages AWS Elemental MediaConvert, which transcodes media and prepares it for on-demand access.

“Text interactions may be the lifeblood of Slack, but we’ve seen greater demand for richer communication options with the rise of work-from-anywhere. Video can help you relay your thoughts more quickly, especially in one-to-many knowledge sharing, and we realized our entire media pipeline needed a revamp to make it shine on the recipient side,” said Slack Senior Staff Technical Lead Manager Kelly Moran. “Slack is invested heavily in AWS, so we knew MediaConvert would be an excellent fit for our transcoding needs, and it allows us to ensure security compliance by using our VPC.”

Before using MediaConvert, clips treated video the same way as other content. Senders uploaded files to Slack, which were stored in an Amazon Simple Storage Service (Amazon S3) bucket and then downloaded by recipients. The entire video file had to be downloaded prior to playback, leading to long buffer times for large files. Client compatibility also proved a challenge as clips did not support file conversion to common media formats or recording in Slack directly. By using MediaConvert to process the video, clip uploads are viewable almost immediately. Uploads are pre-processed as MP4 files, then sent in parallel to AWS Elemental MediaConvert, which generates the HLS output and enables adaptive bitrate (ABR) streaming, and to Amazon Transcribe, which generates line-by-line, time-stamped transcriptions by default. Once the first HLS segment has buffered, which typically takes a few seconds, transcoded, auto-captioned videos are available for viewing.

“We wanted to make files playable as soon as possible, without investing in media infrastructure. With MediaConvert, our streaming playback has been improved by orders of magnitude, and now buffering is negligible to the customer experience,” Moran explained. “We’ve also architected our setup to run transcription in parallel to the transcode for further acceleration. That information is then embedded into the file manifest, which makes the content searchable.”

Improving video quality was another key priority for the engineering team in Slack’s revamping of the clips media pipeline. After experimenting with various resolution, frame rate, and bitrate settings in MediaConvert, the team found a set of configurations that optimized the user experience and costs. In the end, they opted for Quality-Defined Variable Bitrate (QVBR) mode, which automatically adjusts video bitrate to maintain consistent video quality and save bandwidth, and streamlined resolutions to 720p and 1080p, except in cases where the source file has a lower resolution. The player selects the best version to play based on bandwidth and network connection.

Since sharing slide presentations is a key function of clips, any text must be crisp. Previously, clips resolution maxed out at 720p and viewers encountered legibility issues. With the new workflow, files are universally playable, and transcoded to HLS for smooth playback in seconds, a process that might have taken as long as several minutes previously. Clips also now supports the higher 1080p resolution, with little to no impact on time to first frame. Additionally, audio outputs are automatically transcribed via Amazon Transcribe to create captions for increased accessibility and searchability.

“The documentation for MediaConvert was immensely helpful. Our engineering team is mostly generalists, so navigating bitrates and resolutions was new territory for us,” Moran concluded. “With this foundation in place, we can now explore additional ways to further enrich the Slack user experience with media for deeper engagement.”

To learn more about AWS Elemental MediaConvert, please visit: https://aws.amazon.com/mediaconvert/