What an exciting time it was to host the 25th Midwestern Association of Language Testers (MwALT) Conference right here in Pittsburgh, the City of Bridges! This year’s theme, Bridging competing priorities in language assessment: Connecting validity, reliability, and practicality, could not have been more fitting for this gathering of language assessment experts. From thought-provoking presentations to hands-on workshops—and plenty of great food at Duo’s Taqueria!—this year’s event was a highlight for language testing experts from across the Midwest and beyond.
Exploring the balance between validity, reliability, and practicality
MwALT 2024 kicked off with an opening plenary by Dr. Eunice Eunhee Jang from the University of Toronto, who delivered an insightful talk on how the integration of diagnostic and dynamic assessment is reshaping the way we think about language testing. Her talk, Enhancing Learning through the Integration of Diagnostic and Dynamic Assessment via Technological Innovations, showed how real-time data processing and automated scoring systems are bridging gaps in educational access, making assessment a powerful tool for learning.
As we explored the conference’s theme, the presentations and workshops reminded us of the delicate balance that exists between creating assessments that are valid, reliable, and practical. From test developers to university admissions officers, everyone in the language testing community faces competing priorities—and MwALT 2024 provided a space for experts to come together and discuss how to navigate these challenges.
Pre-conference workshops: Hands-on innovation in test development
Ahead of the main conference, attendees participated in two fascinating pre-conference workshops that explored cutting-edge innovations in the world of language testing.
- Workshop 1: Using GPT for Item Piloting and Automated Scoring, led by Alistair Van Moere and Jing Wei from MetaMetrics, offered participants a deep dive into the use of GPT to streamline the item piloting process and assist in rating student responses. Attendees explored how large language models (LLMs) can simulate student interactions and alleviate some of the labor-intensive tasks associated with traditional item development and grading.
- Workshop 2: Using LLMs for Content and Item Generation, led by Andrew Runge, Yena Park, and Basim Baig from Duolingo, took participants through the practical steps of leveraging LLMs to create language assessment items. With hands-on examples and demonstrations, the workshop emphasized how educators and assessment developers can integrate LLM-generated content into their workflow to develop diverse and high-quality assessment items.
These workshops showcased how GPT and LLMs are reshaping the landscape of test development, giving attendees the tools to enhance their own testing practices.
Pushing the boundaries of language testing research
Duolingo was well-represented at this year’s MwALT, with several of our researchers presenting innovative work that pushes the boundaries of language assessment.
- Ben Naismith and Masha Kostromitina presented their paper with Jill Burstein and Luke Plonsky, Beyond GPA and Language Proficiency: Factors of International Students’ Academic Success. Their systematic review examined the diverse factors—beyond language proficiency—that contribute to international students’ success, advocating for a more holistic understanding of academic achievement that takes into account personal and social factors.
- Ramsey Cardwell followed up with a fascinating presentation of his work with Jill Burstein and Steven Nydick, on the The Complex Role of English Language Proficiency in Predicting Academic Outcomes. Their longitudinal study explored the nuanced relationship between English proficiency scores and GPA outcomes for international students in U.S. universities. This research sheds light on the role AI-powered assessments play in predicting student success and the complex factors influencing these outcomes.
Both presentations offered a deep dive into the multifaceted nature of academic success for international students and the important role that English language proficiency plays.
Fun, festivities, and celebrating emerging scholars
It wouldn’t be a Duolingo-hosted event without some fun! Conference attendees got a taste of Pittsburgh culture (and some seriously delicious tacos) at the opening reception, held at Duo’s Taqueria. Set in Duolingo’s own restaurant, the reception gave attendees a chance to mingle, relax, and enjoy an evening of good food and conversation.
The conference wrapped up with a memorable closing banquet, held in the heart of Pittsburgh. After newly-appointed MwALT President Mark Chapman thanked the crowd for their enthusiastic participation at the conference, before MwALT member at large Fabiana MacMillan announced this year’s student award winners!
First up was Burak Senel, winner of this year’s Best Student Paper Award, for his outstanding paper Lexical Bundles as Predictors in AI-Powered Scoring of Non-Native Academic Writing. Burak’s work exemplifies the innovative research that continues to push the boundaries of language testing.
Then, Ping-Ling Chuang was named the winner of this year’s Best Student Presentation award, for her excellent presentation, Eye-tracking insights into rater behavior: Comparing experienced and novice raters in L2 writing assessment. Ping-Ling’s research offered fascinating insights into how different levels of experience influence rater behavior, contributing valuable knowledge to the field of writing assessment.
Thank you to everyone who made MwALT ‘24 a success!
We’re incredibly grateful to all the presenters, attendees, and organizers who made MwALT 2024 such a rewarding and thought-provoking experience. A special thank you to our plenary speakers, workshop facilitators, and the conference planning team. As we move forward, we’re excited to continue bridging the gap between innovation and practicality in language assessment.
We’re already looking forward to next year’s conference, and we hope to see you all there! To learn more about Duolingo’s contributions to language testing research, check out our publications page.
