Cutting Teacher Grading Time in Half Using Amazon Bedrock with Code.org

Code.org provides K–12 students with access to computer science and AI learning opportunities at no cost. “Almost every K–12 student learns about photosynthesis, but not every student grows up to be a botanist,” says Suresh Chanmugam, software engineer at Code.org. “We should think of computer science the same way. Understanding how it works and learning how to create programs is important for understanding our place in society.”

Since its founding in 2013, Code.org has used AWS. Though the organization’s initial core technology was relatively simple, it knew that having the support of a reliable, cost-effective, and scalable technology foundation would accelerate its growth, which nearly tripled in 2015–2023. As the organization expanded and created new resources and offerings, it harnessed more and more AWS services and features, seeing the potential of AI to better support teachers’ needs.

Code.org’s primary goal was to save time for teachers who regularly spend hours grading computer science and coding assignments—especially new teachers, who often reported feeling less confident in their capability to catch coding errors. At the same time, the organization wanted to preserve teachers’ agency and leadership by maintaining their responsibility for making final grading decisions.

Therefore, Code.org needed to design a model to provide qualitative feedback that teachers could review and update before sharing with students. “We don’t believe that computers alone can teach children; we’re a very teacher-centric organization,” says Simon Guest, chief technology officer at Code.org. “Supporting teachers is just as important to us as supporting millions of students around the world. Using AI on AWS, we have successfully addressed teachers’ needs.” Finally, Code.org required its solution to protect students’ personally identifiable information, because many coding projects include names and other sensitive details.

To build and train its responsible, unbiased AI Teaching Assistant tool, Code.org used Amazon Bedrock, a fully managed service that offers a simple way to build and scale generative AI applications. This gave Code.org the freedom to experiment with a wide range of foundation models and fine-tune its selections. After testing several options, the organization chose to move forward with Anthropic’s Claude 3 Sonnet model. “We considered problem-solving and evaluation criteria as we tested a number of models,” says Guest. “Claude 3 proved exceptionally useful for a teacher.”

Code.org developed its AI Teaching Assistant in 9 months. The tool offers a qualitative assessment that is based on an assignment rubric—a guide for scoring student submissions. Then, it provides teachers with precise mappings of where a student’s work falls on the rubric. Teachers can use this as a baseline when making their final grading decisions.

When students submit assignments, the tool runs automatically in the background to reduce waiting time while the AI assistant evaluates the work. This also prevents the tool from impacting Code.org’s latency. The AI assistant takes up to 1 minute to complete a job, but by running jobs asynchronously, educators don’t have to wait for results when they’re ready to start grading. “We want to complete jobs as quickly as we can,” says Darin Webb, infrastructure engineering manager at Code.org. “By the time the teacher gets there to take a look at it, it’s done.”

After a pilot period, Code.org used teacher feedback to improve the model’s accuracy and then rolled out the solution to up to 2,000 teachers. The benefits were clear right away. Users reported a reduction of up to 50 percent in grading time, so they could dedicate more time to supporting their students, providing personalized feedback, and building lesson plans. Using the AI Teaching Assistant, teachers also reported feeling more confident about their own judgments and decisions. One teacher said that it gave another layer of validity to the assessment in the eyes of students, taking human error and bias out of the equation.

Code.org’s AI Teaching Assistant harnesses the power of AWS to improve the educational experience for both teachers and students. Now that teachers can spend less time grading assignments, they can better provide students with personalized support and higher-quality, unbiased feedback.

Code.org plans to continuously improve its AI tool. It is already using AWS to experiment with additional AI tools and features, including a chat experience that helps teachers update lesson plans and tailor activities and assignments to individual students’ needs. The organization is also looking forward to offering students opportunities to learn about AI. Plus, it is building features for customizing learning paths and teacher interventions to students’ knowledge and performance.

“Using AWS over the years gave us confidence that we could be successful with AI and get support when we needed it to unlock new solutions,” says Webb.