Five years ago, we launched the first high-stakes, digital-first test that could be taken anytime and anywhere in the world, because we saw how technology could radically improve testing for students and institutions. In this five part series, we take a look back at the enormous amount of research and development that went into reinventing the world of high-stakes testing.

Securing an appointment and sitting for a lengthy exam at a test center is inconvenient and expensive—not to mention that for millions of people across the globe, simply getting to and from an appointment costs far more than the test itself.

At Duolingo we are committed to eliminating barriers to education. Test centers exist because the old model placed more importance on security than on access. Luckily, using today’s technology, we can have both.

Read on to learn how our experts leverage the latest tech at every stage of proctoring to ensure the security of the Duolingo English Test.

A new approach

“From the first day the test was created, it was always going to be a digital test—never in person,” says Basim Baig, Head of Security for the Duolingo English Test. “That’s why we’ve thought long and hard about how to do digital proctoring really well.”

Baig explains that the goal in designing Duolingo English Test security was not to replicate the old testing center model, which has several security flaws and acts as a barrier to many students.

In an average testing center, one proctor supervises over 25 test takers at a time. Proctors are only able to oversee the security of the exam while each test taker is on site; a single person breaking a rule can invalidate hundreds of exams taken at the same location. And then there’s the potential conflict of interest: proctors are often contracted locally, so they are vulnerable to threats, bribes, or other forms of influence.

Breaking free from the test center model makes it possible not only to improve access to testing, but also to mitigate these security flaws.

Testing on your terms

You can take the Duolingo English Test whenever and wherever you choose--all you need is an hour, an internet connection, and a quiet room. This way, you can test when and where you’re at your best, free from the distractions of a traditional test center.

The Duolingo English Test desktop app 

Testing takes place on the Duolingo English Test desktop app, which is packed with a host of security features that help deter malicious rule breaking during the test. The app is in full screen mode for the entirety of the test session. No add-ons or third-party apps are allowed; accessing other tabs or toolbars will stop the test immediately.

“Of course, with any test, it is possible that people will attempt to break the rules,” says Baig. “Using AI, we detect these attempts at multiple stages—during the test, and during proctoring after the fact.”

Once a test is completed, asynchronous digital proctoring allows for multiple independent rounds of test session review, in which teams of anonymous proctors evaluate each test session.

Comprehensive proctoring

Our innovative, comprehensive approach to proctoring can be understood as human-in-the-loop AI; we use machines and humans in places they both perform best, always leaving final decisions to the humans.

Traditional proctoring vs. our approach

First, our AI algorithm conducts an automated review, looking for potential signs of rule breaking or malicious behavior. At this stage, we leverage a sophisticated combination of statistics, computer vision, and natural language processing algorithms to analyze over 75 different behavioral and environmental factors.

When an algorithm raises a flag on a potential security breach, human proctors analyze the warning on the test in question to determine whether any rules have been broken. Because each test session is discretely proctored, rule breaking by an individual test-taker does not impact the validity of other tests. To ensure that proctors are not subject to bias, they are anonymous at every round of review.

After a session has been reviewed by multiple human proctors and a certification decision is reached, the test taker is informed of their results and they can share their scores with institutions. Thanks to AI, this entire process takes only 48 hours.

A win-win

Beyond solving the problem of test centers limiting access, the Duolingo English Test’s innovative, digital-first approach to proctoring makes our English test more secure than those taken in traditional test centers.

“In leveraging the tech to prioritize both rigor and access, we’re able to have our cake and eat it too,” says Baig. “What we've done is make one dimension better, while also vastly improving another dimension. That’s like magic, that doesn't happen all the time!”