Although the landscape of international education is constantly evolving, a crucial question for admissions officers and prospective students persists: How accurately do English proficiency test scores predict academic performance?
At Duolingo, we're committed to answering this question through rigorous empirical research. Our latest studies on the Duolingo English Test offer compelling insights into its predictive validity—that is, how well it predicts future outcomes. Turns out DET test-takers perform just as well as students admitted with other widely accepted tests!
How strong is the Duolingo English Test’s predictive validity?
In the context of language proficiency tests, predictive validity refers to how well a test score predicts a test taker’s future performance in a college or university setting. For English language proficiency tests like the DET, predictive validity studies typically examine how well test scores predict academic performance of international students in a college or university setting. This is commonly measured with correlations between test scores and GPA.
To examine the DET predictive validity evidence, our research team partnered with four institutions, each located in the United States: two large public universities, one medium private institution, and one small private institution, analyzing data from over 700 undergraduate and graduate students. These institutional partners provided us with students’ DET, IELTS, and TOEFL scores, their GPA, degree level, first language, and country of origin.
The overall findings reveal that the Duolingo English Test demonstrates strong predictive validity. Here are some key findings:
- Positive Correlations: Across institutions and study levels, we found positive correlations between DET scores and student GPAs. This indicates that higher DET scores are associated with higher academic performance.
- Comparable Performance: When comparing DET test-takers to those who took IELTS or TOEFL, we found that DET students performed just as well academically. The average GPAs for DET test-takers ranged from 3.48 to 3.72, compared to 3.47 to 3.75 for IELTS and 3.50 to 3.78 for TOEFL test-takers. There were no statistically significant differences in average GPAs across the three tests.
- Consistent Results Across Institutions: This pattern held true across different types of institutions, from small private colleges to large public universities.
To expand our data pool, we collected additional transcripts directly from students who had taken the DET and were enrolled in higher education institutions across six English-speaking countries. We received 313 usable transcript submissions from 193 universities, including 120 US universities.
Our analysis of data collected in this round supported the trends we detected previously. Both undergraduate and graduate DET test-takers demonstrated strong academic performance as indicated by their high average GPAs. We also found positive correlations between DET scores and GPA for both of these degree levels. These results are summarized in the table below.
Taken together, these findings underscore that the DET is on par with other widely accepted English proficiency tests in predicting international students’ academic success.
Beyond test scores: a holistic view of student success
While our research supports the DET's predictive validity, it also highlights the complex nature of academic performance. In addition to collecting students’ scores and transcripts, we asked students to respond to a survey about language proficiency and other factors impacting their academic performance. Their answers revealed that across all proficiency levels, students face challenges that seem to impact their grades. Some of the themes we identified in students’ responses included the following:
- Emotional and physical well-being
- Additional responsibilities outside of academics
- Barriers to studying or completing assignments
Interestingly, undergraduate students with lower DET scores and GPAs reported experiencing these challenges more frequently. This underscores the importance of support systems and resources for international students, beyond just language support.
Student success depends on many factors!
A final word on predictive validity: it's important to note that correlations between test scores and GPA are generally expected to be modest (e.g., Gagen and Faez, 2024) . This is because language proficiency, while crucial, is just one of many factors that contribute to academic success. In fact, research has consistently shown that other variables such as motivation, study habits, prior knowledge of the subject matter, and even personal circumstances all play significant roles in determining a student's GPA (e.g., Ihlenfeldt and Rios, 2023).
At Duolingo, we're dedicated to continuous improvement and transparency. Our research on the DET's predictive validity is ongoing, with plans to expand our data pool and update our analyses regularly. As we continue this work, we're committed to sharing our findings with the broader education community. For a more detailed report on the findings of the study, look out for our presentation at MwALT 2024, as well as other updates right here on the DET blog!