AWS Partner Network (APN) Blog
Rebuilding an EdTech Publishing Platform by Leveraging AWS Lambda to Optimize Performance at Scale
By Phil Horwitz, CTO – JBS Custom Software Solutions
By Praneeth Perna, Sr. Solutions Architect – AWS
 |
| JBS Solutions |
 |
An influx of technology is rapidly opening up new avenues to digitally enrich the traditional classroom experience. The growing EdTech industry consistently envisions and develops new ways to create more engaging, inclusive, and individualized experiences when it comes to learning.
As part of the industry’s innovation, digital textbooks, especially for the K-12 classroom, are becoming a more prevalent component of technology-based education. These digital textbooks—along with the large platforms that publish them online—offer an array of value-added benefits over the traditional paper or printed version.
Digital textbooks, instantly delivered through the cloud, are accessible anywhere, portable, and compatible with almost all devices—leaving virtually no downtime for learning. They are highly interactive, customizable, and self-directed.
Moreover, digitally-delivered books can be automatically updated at any time while remaining cost-effective for school districts, environmentally friendly, and requiring no physical storage space. The handful of online publishing platforms that deliver these digital textbooks typically offer hundreds of titles to thousands of students, parents, and educators accessing them at any given time.
In this post, JBS Custom Software Solutions (JBS Solutions) shows how it rebuilt one of the largest digital textbook publishing platforms from the ground up. The primary solution leveraged AWS Lambda to optimize performance at scale while providing flexibility along with robust functionality for both the back and front end of the platform.
JBS Solutions is an AWS Advanced Tier Services Partner with the AWS Lambda service delivery designation. JBS Solutions builds enterprise-class custom application solutions to accelerate business innovation.
Use Case Challenges
In a highly competitive process, JBS Solutions was selected to partner with a major EdTech organization that had recently acquired a large, well-established digital textbook publishing platform with titles for grades K-12.
However, it soon became apparent that this platform—containing hundreds of textbooks and serving thousands of users at contracted school districts across the United States—was experiencing significant performance-related problems and challenges:
- The existing platform could no longer support its prodigiously growing volume of customers.
- As originally architected, the platform was unable to accommodate scaling and growth for both the front (customers) and back end (staff).
- It was increasingly experiencing site outages, slowness, and overall inadequate performance affecting all areas and aspects of the product.
- There was difficulty adding new functionality to the platform as well as correcting existing bugs.
- Deploying new code for the outmoded design and architecture was problematic and time consuming.
- New digital book releases were taking too long to develop for the platform and needed to be streamlined via technology to remain competitive in the industry.
This outdated Java-based publishing platform—with a monolithic architecture—had originally been hosted on premises. However, by the time JBS Solutions was engaged, it had been “migrated” to the cloud in a non-cloud-native fashion.
Infrastructure was deployed on dedicated Amazon Elastic Compute Cloud (Amazon EC2) instances. No cloud-native services were used such as Amazon Relational Database Service (Amazon RDS).
Solution Overview
Based on the situation, JBS Solutions and its EdTech customer decided the best course of action would be to build from the ground up an entirely new digital textbook publishing platform in the cloud.
A microservices architecture with independently deployable services would provide the framework for the new platform. This greenfield solution would be fully cloud-native and leverage a serverless structure using AWS Lambda to optimize scaling, flexibility, cost, and effort.
The customer also wanted the new platform to have additional functionality (front and back end) and be more agile so it could keep up with changes to the developing digital publishing landscape. This was important given the complex and evolving regulatory requirements for the Common Core State Standards launched in 2009, along with individual state exceptions.
Functionality was generally expected to leverage all digital aspects of the platform. In other words, it should provide added value to digital textbooks that could never be gotten from a paper or printed version of the book.
Also, the automation of time-consuming and labor-intensive processes for publishing and updating digital textbooks was considered a high priority business requirement.
Some of the new digital functionality required for both front- and back-end use included:
- Automating the entire workflow process and compilation of new and updated digital textbooks which were still using extensible markup language (XML)—provided by a legacy vendor—to describe the contents and layout of a book.
- Processing images, video, audio, and other multimedia components in a systematic and automated manner.
- Allowing editors and staff to easily make on-the-fly adjustments or changes to the content of any digital book in the platform’s catalog.
- Integrating interactive testing content and assessment materials developed by a third-party vendor.
- Tracking and managing the scoring of digital textbook assessments associated with a specific student, class, and teacher.
Technical Aspects of the Solution
The concept behind the technical solution was simply to let the cloud do what it does best. With that mindset, JBS Solutions went all in with a serverless structure built entirely in the cloud using microservices offered by AWS.
The solution focused its efforts on scale and performance, backend automation of processes, and content integration with a third-party vendor.
Scale and Performance
The biggest obstacle facing the legacy textbook publishing platform was its inability to scale out horizontally to increase its overall capacity to accommodate growing traffic and new functionality.
This made AWS Lambda the perfect solution for addressing overall scale and performance. Lambda’s elasticity or ability to “stretch”—concurrently with the publishing platform’s increased resource usage and enhanced functionality (both front and back end)—made it an ideal use case.
While the monolithic code for the original platform was Java-based, JBS Solutions coded the new application, with all its enhanced functionality, entirely in Python using the Django framework so it would have a clean and pragmatic design.
The new code was then run entirely in the cloud using Lambda, which automatically scales out horizontally with no further intervention needed.
Both the scalability and ephemeral nature of Lambda—where it shows up on the fly, efficiently runs code, and then quietly disappears—addressed the negative performance issues endemic to the legacy publishing platform.
In short, with an AWS Lambda serverless architecture the EdTech customer could enjoy all of the performance benefits of horizontal scaling without the operational overhead—paying for only as much as needed while fully optimizing the performance of its new platform.
No more outages, slowness, or inadequate performance. Based on metrics, performance (in terms of time to handle requests and number of requests) has improved by over 120%. In addition, the customer can roll out new functionality and features up to 25x faster due to moving from a deployment schedule of every six months to a weekly schedule.
Due to the lower operational overhead of the solution, production support tickets have decreased by at least 70%; tickets that do come in are also resolved at least 50% faster.
Backend Automation of Processes
The process for assembling all content to publish a new digital textbook or update an existing one is quite cumbersome and time consuming. From a business perspective, the EdTech customer needed to streamline this entire process so they could offer new book releases faster to remain competitive in the market.
To decrease the time and effort it took to assemble all content for a digital textbook, JBS Solutions provided backend automation to process all XML and multimedia files required to produce a platform-ready digital textbook.
The content for digital textbooks came from a document management vendor who prepared the electronic preprocessed documents from a print-based format using XML. This existing process could not be changed or modified. The markup delivered fully described the textual content of a book while clearly identifying all its sections and parts.
A complete workflow process hosted on AWS Lambda was developed by JBS Solutions to automate almost all tasks necessary for compiling the contents of digital textbooks provided in XML. This workflow takes the raw XML and transforms it into the actual pages of a book that can be displayed to the front-end user of the platform—students, parents, and teachers—using almost any device.
Because related images and multimedia content for textbooks were delivered separately from XML files, these objects were initially placed into Amazon Simple Storage Service (Amazon S3) buckets. The occurrence of certain S3 events would automatically trigger a Lambda function to process them.
As part of the automation and transformation process, images and multimedia files were given their own special processing pipelines that fully leveraged the computing power of AWS Lambda. For example, raw images had to be adjusted for the correct format and size, metadata needed to be tagged to each image, and an association between the image and where it needed to be placed in a textbook. All of this was orchestrated by Lambda.
Similarly, Lambda-based processing pipelines were also developed by JBS Solutions for video, audio, and other multimedia file automation.
Lastly, JBS Solutions provided a separate application using Lambda that would allow any book editor on the back end of the publishing platform to make quick updates or on-the-fly edits to the content of any digital textbook in the platform’s vast catalog.
Content Integration with Third-Party Vendor
To take advantage of the digital aspect of textbooks, the EdTech customer partnered with a third-party content creator that specializes in developing interactive tests, quizzes, and assessment materials.
Interactive assessments and test content were considered a critical element for delivering added value to the digital textbooks on the customer’s platform. The content provided by the vendor for these interactive tests and assessments were processed through automated pipelines using AWS Lambda. This was necessary to properly place the testing materials into the correct location of a corresponding textbook.
JBS Solutions coded all applications for delivering student assessments on the front end, along with the necessary tools to fully track and manage test scoring on the back end of the platform. All test results and assessments could be directly associated with a specific student, class, teacher, and textbook.
Figure 1 – Architecture for new digital publishing platform.
Conclusion
This post explored and unpacked the ways that digital textbook platforms are impacting technology-based education all the way from the cloud.
With insights into a major digital textbook publishing platform, we showed how AWS Lambda was leveraged to optimize functionality and performance at scale. Performance involved both the delivery of the digital textbooks to students, parents, and teachers (front end) and automating the process for assembling the contents of these textbooks (back end)—a major business requirement for reducing time-to-market and remaining competitive in the industry.
We also showed how Lambda could effectively automate the process of integrating testing and assessment content developed by a third-party vendor into corresponding digital textbooks.
This platform solution developed from the ground up by JBS Custom Software Solutions and will be used by thousands of students and educators into the future. Visit the JBS Solutions website for more details on how it is delivers leading-edge solutions for business.
.
.
JBS Custom Software Solutions – AWS Partner Spotlight
JBS Solutions is an AWS Advanced Tier Services Partner that has been delivering leading-edge software and application-based solutions for clients for over 20 years to support proprietary business models and innovative technological solutions.