Penn State’s Data Empowered Learning team spent a great deal of time working with faculty to understand their student support, course delivery, and pedagogical needs, seeking to understand what questions they ask themselves about the courses they teach. From these discussions, the team learned that instructors had a broad need to understand the demographics of the population of students enrolled in their courses—for example, data about students’ previously enrolled courses could help instructors adapt their teaching approach each semester based on students’ level of understanding going in.
Instructors also said it was important for the tool to be accessible through their existing Canvas dashboard. Hellar says, “We received feedback from instructors that told us, ‘I don’t want to use another tool—I live in Canvas—that is where I teach. So if you’re going to give me any new analytic data, put it in Canvas.’” Penn State was able to build Course Insights using Instructure’s detailed documentation for Canvas and its support for learning tools interoperability (LTI).
The Course Insights pilot rolled out to a small number of faculty members in the fall of 2021 and has grown steadily since then. Although Penn State built the tool in-house, the team meets regularly with Instructure and interacts with the company almost daily. “Penn State partners really closely with us,” says Melissa Loble, chief academic officer at Instructure. “Our open, standards-first approach, alongside our detailed documentation and very active community, helped to enable the work that Penn State has done with Canvas—and we’re there to support them whenever they need us.”
Since launching as a limited-access pilot, Course Insights has been rolled out to more than 250 instructors who teach over 4,000 courses. To provide instructors with data insights at scale for every canvas course and every enrolled student at Penn State—up to 89,000 students—the tool analyzes 2 TB of learning activity data every day from sources including Canvas, Kaltura, Packback, TopHat, and other learning tools, and also includes advising data from Starfish and student data from the university’s Student Information System (SIS).
The data is stored using Amazon Simple Storage Service (Amazon S3), object storage built to retrieve any amount of data from anywhere, and processed using Amazon Elastic Compute Cloud (Amazon EC2), which offers secure and resizable compute capacity for virtually any workload, with instances spun up and down according to need. Using Course Insights, instructors can see a comprehensive view of learning activity from the supported learning tools showing a rolling 7-day average of student activity in each of their courses. Anomalous activity is highlighted—orange for periods of zero activity, yellow for activity levels that have fallen below 50 percent of the course average activity of their peers—making it easy for instructors to identify students who need immediate support to reduce the chances of failing or dropping out of class.
Thanks to Course Insights, the Data Empowered Learning Team of Penn State Information Technology won a Platinum 2023 Learning Impact Award from the 1EdTech Consortium. “This award helps validate that our approach of delivering learning analytic data to instructors has value not only to the institution but for the industry at large,” says Hellar. And Instructure itself is building on the success that Penn State has had with Course Insights. “The incredible work they’ve done has influenced where we’ve taken Canvas data and how we think about our documentation, our standards, our APIs,” says Loble. “We’ve used this to continue to inform our roadmap.” Instructure also updated Canvas in late 2023 to reduce the application’s latency—previously, the data displayed could be more than 24 hours old, but the new version brings reporting closer to real time.