In 2023, the Duolingo English Test hosted its first research conference in Asia, at the Shangri La in Hong Kong! With a formidable lineup of 22 invited speakers and experts in the field of language assessment, learning, and teaching, the conference provided a visionary look at the future of language assessment and the integration of AI technologies.
A meeting of the minds
Organized together with Qin Xie from the Education University of Hong Kong and Edward Wen from Hong Kong Shue Yan University, with Antony Kunnan serving as conference adviser, the conference brought together esteemed scholars from across the Asia region to discuss pressing topics and new research in language assessment.
Jun Liu, the Rector of City University of Macau, launched the keynote speeches with his talk, "The Nexus of Teaching, Learning and Assessment of English Language in the Era of AI." Reflecting on his 2001 work on Asian students' classroom communication patterns, Liu discussed how the assessment of linguistic competence has evolved in the context of World Englishes. He highlighted the intersections and interrelationships of teaching, learning, and assessment, arguing for the need to leverage technologies in the AI era to provide inclusive learning experiences that embrace diverse English language varieties.
Jianda Liu, a full professor and the Dean of the National Key Research Center for Linguistics and Applied Linguistics at Guangdong University of Foreign Studies, delivered the first plenary session on "The Application of ChatGPT in Classroom Assessment." His talk focused on the transformative role of ChatGPT in classroom language assessment, providing innovative ways to evaluate students' language skills and enhance their learning experience.
Featured speaker David D. Qian, Emeritus Professor of Applied Linguistics and Language Assessment and Deputy Director of the Research Centre for Professional Communication in English at Hong Kong Polytechnic University, explored the challenges and opportunities posed by generative AI tools in educational landscapes in his talk, "Responding to the Emergence of Generative AI on the University Campus." He reviewed institutional policies and examined teachers' attitudes and actions in dealing with AI-related issues, pinpointing key areas for reshaping the educational system in the AI integration context.
A notable contribution came from Xiaoming Xi, Director of Assessment and Technology at the Hong Kong Examinations and Assessment Authority. Xi highlighted the importance of returning to the fundamentals of test validity and construct in language testing for university admissions, amidst the growing influence of AI technologies and the changing landscape due to the pandemic.
Wrapping up the plenaries, Liying Cheng, Professor and Dean of the School of Education at the City University of Macau, presented "Catalyzing an Ecosystem for Language Assessment, Learning, and Teaching." She emphasized the need to reconceptualize the symbiotic relationships among assessment, learning, and teaching in the face of the rapid advancement of AI and machine learning. Cheng proposed a vision for an educational ecosystem where humans and machines interact in a mutually supportive environment.
Innovations and challenges
The hottest topic at the conference was AI. A panel of experts representing language centers in major universities in Hong Kong and Macau explored the exciting possibilities that AI brings to language assessment in a roundtable forum focusing on "The Impact of ChatGPT," conveyed by Dr. Zhisheng Wen and moderated by Dr. Di Zou.
The panelists shared their insights and perspectives on the transformative potential of ChatGPT in education and the corresponding policies adopted by their respective universities in association with the challenges and opportunities brought about by AI technologies. Their discussion touched upon the issues of bias, privacy, academic honesty and other ethical considerations. The panelists shared their strategies for addressing these challenges, underlining the need for a careful, balanced approach to AI integration. The forum was wrapped with a number of implications of AI for the future of language education and the skills required for language educators.
But the conversation wasn't limited to potential hurdles. The panelists also discussed how this technology can enhance language learning and assessment practices, enable personalized and adaptive learning experiences, and provide valuable feedback to students. The forum was wrapped with a number of implications of AI for the future of language education and the skills required for language educators.
Representing the Duolingo English Test at the conference were Lead Assessment Scientist Geoff LaFlair, and Senior Assessment Scientist Mancy Liao. Mancy’s talk, "Quality Assurance in Digital-First High-Stakes Language Assessments” spotlighted the unique challenges posed by the flexibility, complexity, and high-stakes nature of digital-first language assessments. Liao illustrated the development of the Analytics for Quality Assurance in Assessment (AQuAA) system, a crucial tool for anomaly detection in high-stakes language assessment.
In his presentation "Leveraging Large Language Models for Complex Listening and Reading Tasks in Language Assessment: The Duolingo English Test Experience," Geoff discussed the transformative impact of large language models (LLMs) on content and item creation for language tests, enabling the development of complex listening and reading tasks. His talk highlighted the human-in-the-loop approach to ensure quality and fairness in automated content generation using LLMs.
Broad horizons
Although this was primarily an assessment conference, we branched out into various exciting avenues of research in applied linguistics.
Take, for instance, MingMing Chu's intriguing presentation on the statistical learning of conversational structures. Chu delved into the intricate world of conversational dynamics, exploring how statistical models can be leveraged to better understand and predict dialogue patterns.
Andrew Moody’s talk on World Englishes opened our eyes to the global variations and richness of the English language, demonstrating how language evolves and adapts in different cultural contexts.
Another highlight was the joint presentation by Mathew Wallace and Edward Wen. They gave us an insightful glimpse into the realm of working memory, discussing its role and implications in language learning and comprehension.
Li Ping took us on a fascinating journey through the world of virtual reality. He demonstrated how this cutting-edge technology could revolutionize language learning and assessment, providing a more immersive and interactive experience.
A new era of language assessment
The sheer variety of talks at the conference exemplifies our community’s holistic approach to informing the future of language assessment and testing. By exploring a wide range of research areas and broadening our perspectives, we are constantly pushing the boundaries of what is possible in language learning and assessment!
For more information about the conference and future events, please visit: https://inno-in-lang-assess-2023.vercel.app/programme