As large language models (LLMs) like ChatGPT become increasingly sophisticated, their potential for misuse in high-stakes scenarios has grown. For English proficiency tests like the Duolingo English Test (DET), the rise of LLMs has introduced new challenges in ensuring the integrity of open-ended writing tasks.
To address this, our researchers recently published a paper titled "Detecting LLM-Assisted Cheating on Open-Ended Writing Tasks on Language Proficiency Tests," which was presented at the 2024 Conference on Empirical Methods in Natural Language Processing. The study outlines an innovative framework for detecting responses generated or aided by LLMs, even when modified by test takers.
Here’s how this research is advancing the fight against LLM-assisted cheating.
The challenge: LLM misuse in open-ended writing tasks
Writing tasks are a cornerstone of language proficiency assessments. They provide insight into a test taker’s ability to articulate ideas, construct arguments, and demonstrate language skills in context. However, with LLMs like GPT-4 now capable of generating high-quality text, there’s a risk that test takers may use these tools to cheat.
This problem is compounded by the security measures in place: the DET disables copy-pasting, meaning test takers using LLMs must manually copy-type the generated responses. This process introduces typos, edits, and other modifications, making it harder for traditional detection methods to flag suspicious responses.
Building a better detection system
The research team developed a multi-faceted framework to detect LLM-assisted cheating, even when responses have been manually modified. The approach incorporates several innovative techniques:
- Simulating real-world scenarios:The researchers created a dataset of responses that mimic real-world LLM-assisted cheating. This involved generating text using GPT-4 and introducing copy-typing errors, such as typos, omissions, and word substitutions, to reflect how test takers might manually transcribe LLM-generated content.
- Contrastive learning for robustness:To train the detection model, the team used contrastive learning—a technique that enhances the system’s ability to distinguish between human-written and LLM-generated text. By being robust towards the differences introduced during manual transcription, the model becomes more effective at identifying copy-typed responses.
- Self-training for real-world applications:The framework also includes a self-training component, which leverages responses from real-world test sessions flagged for potential violations. By pseudo-labeling these samples and incorporating them into the training process, the model learns to adapt to emerging patterns of cheating.
Our enhanced model outperforms traditional detection tools
The study demonstrated that the enhanced detection model significantly outperforms existing tools in identifying LLM-generated content, even when manually modified. Key results include:
- A 1.7x improvement in detection rates compared to traditional transformer-based classifiers.
- Robust performance at extremely low false positive rates (0.1%), a critical metric for high-stakes testing environments.
- A consistent upward trend in the rate of detected LLM-generated content over time among test sessions with potential violations, implying an increasing prevalence of LLM being misused to assist cheating.
This research has broad implications for the future of language assessment and academic integrity. By developing detection systems that account for real-world behaviors, we can ensure that test scores remain a reliable measure of English proficiency, even as technology evolves. The findings also highlight the importance of combining AI-based detection methods with human oversight to minimize false positives and maintain fairness.
A collaborative effort
The fight against LLM-assisted cheating is an ongoing challenge, and this research is just one piece of the puzzle. At Duolingo, we’re committed to transparency and collaboration in addressing these issues. By sharing our findings with the broader academic and testing communities, we hope to encourage innovation and ensure the integrity of language assessments worldwide.
To read the full paper, visit our Research and Publications page!