Ever wondered why some students ace English grammar but trip over pronunciation, while others nail vocabulary but struggle with everyday conversation? It's like each learner has their own unique mix of strengths and weaknesses.
As a language educator, I've seen plenty of students with such “jagged” profiles. Because we observe these uneven learning profiles so commonly in language testing, we need to understand how to maintain a delightful experience for test takers with jagged profiles and, more importantly, how to use their test results to inform our instruction.
Some would say that it is simply a matter of students’ individual differences, but there’s certainly more to it. So what exactly defines a jagged profile, and why do some learners exhibit this pattern of uneven progress?
What are jagged language profiles?
In the context of second language (L2) learning, a jagged language profile refers to the uneven development of language skills in areas such as speaking, listening, reading, and writing. This can manifest differently depending on various factors, including individual differences (e.g., motivation, exposure to a target language), language learning and teaching methods, aptitude, and others. In other words, while some language learners may exhibit a more balanced development across language skills, others may excel in certain language skills while struggling with others, resulting in a profile that appears "jagged" or uneven when plotted across language domains.
Understanding jagged language profiles in L2 acquisition highlights the importance of tailored instruction and testing. From the teaching perspective, recognizing learners' strengths and weaknesses across various language domains allows instructors to provide targeted support and interventions to address specific areas of need, ultimately promoting a more effective language learning experience. But on top of that, acknowledging the variability in language profiles is crucial for language testing in order to provide a more precise and pleasant testing experience and accommodate test takers’ differences in language skills.
Are jagged profiles common?
In a word, yes! Because language assessments, and especially high stakes tests like the DET, usually evaluate multiple language skills at the same time including reading, listening, speaking, and writing, jagged profiles are often detected as a result of such tests.
For example, a test taker might score high in reading comprehension but struggle with writing or speaking tasks, and consequently earn lower scores for those skills. This is a common scenario with test takers who have jagged profiles, because we have long known that production-based skills develop later than perception-based skills in L2 learning (Gass, 1984).
Another common thing we see is that a test taker may perform well in reading tasks but struggle with listening comprehension. That’s because listening in an L2 is a complex process that involves sound recognition, segmenting a live speech stream into sounds that make up words and phrases to then be able to decode the meaning behind speech (Rost in Hinkel, 2005). It is not surprising then that some learners may develop this skill later than other skills like reading comprehension.
How can tests address jagged profiles?
Addressing jagged language profiles in English proficiency testing requires a tailored and multifaceted approach. One way to achieve this is crafting a comprehensive, computer adaptive assessment that provides refined measures of reading, writing, listening, and speaking skills.
Thanks to the computer adaptive test administration, the DET adjusts the difficulty level of questions based on the test taker's responses, providing a more pleasant testing experience for learners with jagged profiles. If listening is one’s weak spot, they won’t see C2-level Interactive Listening items that would surely overwhelm them and cause unnecessary test anxiety. Similarly, a learner won’t see a shorter, easier text if they can confidently read complex narratives on general academic subjects.
In order to collect as much information as possible about the proficiency of learners with jagged profiles, a language test would benefit from a variety of task types. The DET is able to tap into test-takers’ strengths and weaknesses in their language ability thanks to its varied task types. For example, instead of relying solely on written production tasks, the test incorporates speaking tasks as well interactive listening and reading tasks and indirect measures of language ability that measure the underlying English skills.
In addition, for scoring of spoken and written responses, tests should be guided by clear scoring rubrics that are designed to account for the nuances of jagged language profiles. At Duolingo, we designed CEFR-aligned rubrics used for grading spoken and written responses and then training our automated scoring models on these grades. In creating these rubrics, we considered differences in the use of lexico-grammatical features that learners at different stages of speaking or writing development may display.
Jagged profiles tell us where to focus our efforts!
By implementing these strategies, among others, English proficiency tests can effectively address jagged language profiles and promote meaningful language development and inclusivity, empowering learners to achieve their full potential in all language domains. Identifying jagged profiles in English language assessment is important not only for test developers and test takers, but also for language educators. Thanks to language tests that are able to provide refined information about one’s language profile, we can detect areas of strength and weakness in language proficiency, guiding the development of targeted instruction and personalized learning plans.