The Duolingo English Test stands out due to its unique, vertically integrated approach to test administration. Unlike many other high-stakes testing platforms that rely on third-party providers to handle various stages of test development, delivery, and proctoring, the DET manages everything in-house. This allows the team to respond quickly and effectively to any emerging security or operational challenges, a flexibility not often found in more traditional setups. Having complete control over the entire testing process means that the DET can maintain a tighter grip on quality, security, and user experience, which benefits both test takers and institutions alike.
One key component of this vertical integration is the ease with which different departments can collaborate and learn from each other. This cross-departmental learning fosters stronger internal communication and deeper understanding across all teams involved in developing, administering, and scoring the DET. One specific initiative that exemplifies this collaboration is our proctor shadowing program.
What is proctor shadowing?
The DET’s proctoring system is remote and asynchronous. This method differs substantially from traditional, in-person test proctoring, where a live proctor supervises test takers in real time. With remote, asynchronous proctoring, proctors review the test session after it has been completed, carefully analyzing video recordings and test data for signs of misconduct. This process offers flexibility and scalability—and it also demands a higher level of technical and procedural expertise compared to standard proctoring.
Since asynchronous proctoring is not widely understood outside our organization, we recognized the need to introduce our team members to the nuances of this process. That's how proctor shadowing was born—a program designed to educate anyone working on the DET, from engineers to marketing specialists, about how our proctoring process works. The goal is to give everyone a firsthand look at how tests are reviewed for integrity, helping bridge gaps in knowledge across the company.
Participating in proctor shadowing is simple. Employees can add their availability to a shared spreadsheet, and they will be paired with an experienced proctor who guides them through the process. During these shadowing sessions, employees observe as proctors review test sessions, flag suspicious behaviors, and ensure compliance with test-taking rules. Team members are encouraged to ask questions, make observations, and fully immerse themselves in understanding the complexities of securing an online, remote exam.
Why do we encourage proctor shadowing?
The proctor shadowing program is more than just a learning opportunity; it's a strategic initiative aimed at improving collaboration and effectiveness across our entire organization. Here are several ways proctor shadowing has benefited different teams within the DET.
Institutional & Government Engagement Team
The members of this team are primarily focused on building and maintaining partnerships with educational institutions, governments, and other organizations that either currently accept or are considering adopting the DET. These stakeholders often have questions or concerns about test security, especially since they may be comparing the DET to other tests that use more traditional proctoring methods.
By participating in proctor shadowing, engagement team members gain a deep understanding of the security measures in place and the steps taken to protect the integrity of the test. This enables them to confidently explain the process to external partners, address any concerns about fairness and security, and ultimately help bolster the reputation of the DET in the global testing community.
Assessment R&D
The research and development team focuses on continuously improving test content and ensuring its validity and fairness. One major challenge this team faces is dealing with attempts to steal or leak test content. Understanding how proctors flag suspicious behavior or potential content theft is crucial for the R&D team. By observing the proctoring process, they can better tailor their content to be more secure and create systems that make it even harder for cheaters to succeed. This feedback loop between proctors and the R&D team strengthens test design and enhances overall security.
Test Taker Markets Team
This team works to understand the needs and behaviors of our test-takers, who come from various countries, backgrounds, and technical skill levels. By shadowing proctors, team members can see firsthand how different types of test-takers approach the DET and what kinds of behaviors are flagged as suspicious.
Understanding the test-taking experience, including what could potentially go wrong, allows the team to provide better assistance and advice to prospective users.This insight helps the team craft better messaging, instructions, and support materials for test-takers, particularly in regions where online testing is still gaining traction.
Security Software Engineers
Our security engineers are constantly working to improve the software and tools used to detect and prevent cheating on the DET. By shadowing proctors, they can gain valuable insights into how cheating attempts manifest during test sessions and what behaviors proctors find most concerning. These observations help engineers refine algorithms and build more robust detection systems, enhancing the security of the test. Direct exposure to the human element of the proctoring process helps engineers design tools that are not only effective, but also user-friendly for the proctors themselves.
The bigger picture: Collaboration and innovation
One of the biggest benefits of proctor shadowing is the way it fosters collaboration across departments that might otherwise operate in silos. When a marketing team member gains insights into the security measures of the test, or when an engineer learns firsthand about the challenges proctors face in real-time, it creates a deeper understanding of how the entire system works. This, in turn, leads to more innovative solutions to problems, better communication across teams, and a stronger, more cohesive organization.