Online Proctoring Renaissance Powered by Artificial Intelligence and Machine Learning

The Chinese Sui Dynasty, (581-618 CE), was the first to establish a nationwide standardized test, The Imperial Examination, to help ensure a common knowledge among state officials and to give imperial rule legitimacy by focusing on merit vs. birth right. This promise of social mobility through education and assessment remains true today.

While the history of exam proctoring is not as well documented as the exam itself, it is safe to say proctoring has always held its place alongside exams to ensure integrity of the results.  Today, we are experiencing a renaissance in how exams are proctored and that revitalization is being created by education technology companies and powered by artificial intelligence (AI) and machine learning (ML).

In 2015, two students from Florida Atlantic University (FAU) set out to level the playing field for online assessments. They devised their own online proctoring prototype and entered the FAU Tech Runway Launch Competition, and won!  The prize money gave them the time and credibility to interview many potential customers – college and university administrators, faculty, and students. These meetings identified the tools and features that would inspire broad confidence in remotely proctored assessments, while furnishing students with a less invasive, less anxiety-ridden testing environment than previously available.  Honorlock was born, and they realized they could be a disruptive force and a beacon of innovation for the education industry.

Honorlock has innovated exam integrity by introducing a browser extension rather than a software download, launching exam content protection technology (Search & Destroy™), detecting secondary devices during exams (Multi-Device Detection™), and providing human voice detection. They have also deployed a hybrid approach to exam proctoring, combining both AI and ML), with live human proctors (Live Proctor Pop-In™). This approach has been shown to reduce anxiety during the testing period.

“As challenging as the remote learning situation has been for the university, there has been a silver lining for us. A partnership with Honorlock has been one of them. Together, University of Florida and Honorlock administered 154,282 exams during the spring semester.”

– Brian Marchman, Director of Distance Learning & Continuing Education, University of Florida

Honorlock’s customers include the University of Massachusetts, University of Wisconsin, Florida State University, Auburn University and Georgia Institute of Technology. With Honorlock, students are now able to take their exams any time day or night.

Honorlock’s hybrid approach using AI, powered by Amazon Rekognition, monitors the student. A live proctor is not watching the exam.  Only when the AI detects suspicious behavior – such as foreign objects, smart phones, new people or voices will a live human proctor intervene, but even then, communication is done via chat box and not face-to-face. Students are stressed about their exam and grade, and having a visible proctor adds to their anxiety.  Honorlock has received feedback from students that a live proctor popping in via chat approach is less stressful than having a stranger watch them during their entire exam.

Honorlock is continuing to research student exam anxiety. Using pre and post-exam surveys, they hope to prove their new proctor training method reduces a student’s test anxiety. Once validated, the AI will be modified to identify demonstrable anxiety. Students exhibiting test anxiety may receive test-taking tips and anxiety-reducing resources and training. In the future they are considering including anxiety detection which includes focus tracking and combines it with question level data (i.e. frequency of changing a response to a test question). They are exploring possibilities of identifying exam keywords that can cause confusion in test-takers so they can create feedback for instructors in order to improve unclear or ambiguous exam questions.

Security is critically important when earning customer trust. To this end, Honorlock is always working to improve their security by being up to date on what it means to be Well Architected and has already taken significant steps to make sure they are secure including: activating AWS Shield for Managed DDOS protection, AWS Web Application Firewall to gain control over how traffic reaches their applications, Amazon Guard Duty for intelligent threat detection. The Honorlock team also takes advantage of AWS Trusted Advisor to improve their overall security posture, the AWS Security Hub to centrally view and manage security alerts and automate security checks, and uses Amazon Detective to both analyze and visualize security data so they can quickly get to the root cause of any potential security issues.

In April of this year, Honorlock joined the AWS EdStart program, enabling them to accelerate their migration to AWS’s AI and ML services. This happened to coincide with most higher education institutions moving to a remote learning and examination model; requiring Honorlock to scale their technical capabilities to new heights. In 2019, Honorlock technology was equipped to accommodate 2,500 concurrent proctored exams, post migration they scaled their technology to accommodate 60,000 concurrent exams and they are now quickly ramping up to deliver over 100,000 exams concurrently. With each exam, Honorlock generates four calls per minute to Amazon Rekognition. With 100,000 concurrent exams there are 24 million Amazon Rekognition calls an hour. Honorlock is also looking to increase the number of Amazon Rekognition features it’s utilizing which will in turn increase the number of calls for each exam.

“The AWS EdStart program has been instrumental in Honorlock’s recent success in a few notable ways. First and foremost, there is the exposure to, and availability of, the subject matter experts the program makes available to AWS EdStart Members. We also found great value and assistance in the AWS EdStart promotional credits which helped to offset AI and ML costs, and allowed us to invest that capital into areas of the business to unlock additional growth opportunities.”

– Q. Wade Billings, Chief Technology Officer, Honorlock

Honorlock also uses Amazon Relational Database Service (Amazon RDS), Amazon Elastic Cloud Compute (Amazon EC2), Amazon ElastiCache to seamlessly set up, run and scale in memory data stores in the cloud, Amazon Simple Queue Service for managing messaging ques for serverless applications, Systems Manager to gain operational insights and take action on AWS resources, and Cloud Formation to speed up cloud provisioning with infrastructure as code.

Honorlock is an AWS EdStart Member. AWS EdStart, the AWS educational technology (EdTech) startup accelerator, is designed to help entrepreneurs build the next generation of online learning, analytics, and campus management solutions in the AWS Cloud. Learn more about AWS EdStart.