Blackboard Optimizes on Amazon EC2, Provides Video Conferencing to Millions of Teachers and Students

Because of the COVID-19 pandemic and the rapid shift to online learning in 2020, education technology company Blackboard, now part of Anthology, needed to scale its capacity to support its virtual classroom solution, Blackboard Collaborate, which had experienced a 4,800 percent increase in usage compared to 2019. Chiefly concerned with providing an interruption-free experience for its users, Blackboard erred on the side of overprovisioning; in the long term, however, the company needed a more cost-efficient solution.

Blackboard developed this solution by making smart use of Amazon Web Services (AWS), including automatic scaling groups and predictive scaling. The company also optimized its use of Amazon Elastic Compute Cloud (Amazon EC2)—a web service that provides secure, resizable compute capacity in the cloud—by switching its preference to instances powered by AMD EPYC processors, which offer cost savings over similar instances; and Amazon EC2 Spot Instances, which enable Blackboard to take advantage of unused Amazon EC2 capacity and save additional costs. The integrated solution enabled Blackboard to scale effectively through the surge in demand, provided the company a 10 percent performance boost, and reduced the costs of its media processing and batch recording compute infrastructure by about 28 percent.

A global education technology company, Blackboard (now part of Anthology) has over 150 million users across higher education institutions, K–12 schools, businesses, and government agencies. When the COVID-19 pandemic halted in-person learning worldwide, Blackboard saw concurrent usage of Blackboard Collaborate increase by 45 times within 4 weeks. “We had entire countries shifting to online learning overnight,” says Kris Stokking, vice president of software engineering at Blackboard. “Not only did we have to accommodate the increased usage, but we also had to support the institutions as they shifted their entire paradigm from onsite to online learning.”

Blackboard had already started modernizing its compute infrastructure 1 year before the start of the COVID-19 pandemic. The company had migrated its mission-critical media servers to Amazon EC2 Auto Scaling, which enables Blackboard to automatically add or remove Amazon EC2 instances according to conditions it defines. The company had also redesigned its recording processing system around AWS Batch, which enables developers, scientists, and engineers to simply and efficiently run hundreds of thousands of batch computing jobs on AWS.

At the beginning of the COVID-19 pandemic, Blackboard was relying on Amazon EC2 C5 Instances, specifically using the Intel-powered c5.9xlarge instances. When faced with the dramatic surge in compute demand, Blackboard took the AWS suggestion to diversify its instance types and avoid the bottleneck that comes with being pinned to a specific instance type and size.

Blackboard found the c5a.8xlarge instances, featuring AMD EPYC processors, to be the ideal fit. Plugging this instance type into the compute environment only involved minor infrastructure reconfiguring using Terraform, testing the existing software on the new instance type, and comparing the performance. No software changes were necessary. “In our testing, we were actually seeing a lower CPU usage for an equivalent load on the AMD instance even though it had a smaller number of CPUs,” says Gavin Llewellyn, senior software engineer at Blackboard. “By switching our preference to AMD-powered instances, we could handle 10 percent additional media processing using equivalent capacity,” adds Chris Cooksey, senior software manager at Blackboard.

Using AMD instances would enable Blackboard to run fewer instances in its Amazon EC2 Auto Scaling group, thus reducing total compute costs, so the company prioritized AMD instances on its list of acceptable instances. Blackboard further optimized its compute environment using AWS predictive scaling with the AMD instances, enabling it to track the number of users on an hourly basis and automate provisioning in accordance with projected requirements.

Blackboard realized further savings by moving some of its workloads to Spot Instances, beginning with postprocessing video work for recordings. Initially, the company had a single AWS Batch compute environment powered by Amazon EC2 On-Demand Instances. The company introduced a second AWS Batch compute environment that prioritized Spot Instances—available at a discount of up to 90 percent compared to On-Demand Instances—and shifted workloads to the On-Demand Instance environment only when Spot Instances were unavailable. “We’ve been running almost 100 percent Spot Instances for recording processing since then, with average price savings of 64 percent in the first month,” says Llewellyn.

Confident in the availability of Spot Instances, Blackboard began using more within its media server Amazon EC2 Auto Scaling group: the mission-critical environment at the core of its audio-video conferencing sessions. The team created scripts that listened to the Amazon EC2 metadata service for notification of when the instances would be reclaimed. AWS offers at least a 2-minute warning for Spot Instance interruption, and with the Capacity Rebalancing feature, Blackboard can be notified proactively when instances are at elevated risk for an interruption, enabling the company to seamlessly shift to On-Demand Instances.

Because Blackboard had already written code for scaling or terminating instances, it was simple to apply similar code to migrate back to On-Demand Instances when Spot Instances became unavailable. Except for a few minor disruptions to its postprocessing video recording, Blackboard faced no problems with its move to Spot Instances. As of May 2021, the company uses Spot Instances for 50 percent of its media servers’ compute needs. “The use of Spot Instances creates some technical complexity that might explain why some engineers are cautious to adopt it,” says Stokking. “But if we’d known earlier what we know now, we would have pursued Spot Instances sooner and more aggressively.”

Using AMD-powered instances and Spot Instances, Blackboard brought Blackboard Collaborate to millions of students and teachers worldwide with minimal disruption—and managed its costs in the process. “AWS was very forthright about where we could cut costs, and that helped build a strong level of trust,” says Cooksey.