We know that interactive tasks are the best way to measure interactive language skills. The DET’s new Interactive Writing task reflects the demands of real-world academic writing, where success often hinges on engaging with a topic in depth, and expanding on related ideas to develop complex arguments.
What is Interactive Writing?
Interactive Writing is a multi-step question type on the DET, designed to reflect the revision process that is a critical part of academic writing, where writers are expected to refine their thoughts and arguments based on feedback or further reflection.
First, test takers are given 5 minutes to respond to a prompt. They may be asked to narrate, evaluate, or persuade: all rhetorical techniques that are critical to academic and professional writing tasks. Then, based on their initial response, test takers receive a follow-up prompt, and are given three minutes to elaborate on the original subject in light of that new prompt.
The two-stage design encourages test takers to reflect on their initial ideas and expand upon them, allowing them to showcase their higher-level writing skills, and increasing the total amount of time devoted to writing on the DET.
What does Interactive Writing measure?
Interactive Writing measures (you guessed it!) writing skills, and contributes to the Literacy and Production subscores test takers receive in their certified score report two days after they take the test. Our assessment scientists worked hand in hand with our machine learning engineers, as well as a writing studies expert, to design this new item.
The two-stage prompt provides what educators call “scaffolding” to guide the test taker’s writing, providing feedback that isn’t critical, but rather expansive. This gives test takers a chance to further develop their response, and allows us to collect longer, more informative writing samples. Incorporating this interactivity expands the test’s writing construct—that is, what the test measures—to include the ability to use content-oriented feedback to further develop one’s own ideas.
Not only does this expand the writing construct, it also reduces construct irrelevant variance, or the extent to which a test’s results are influenced by factors that aren't relevant to the construct. Encouraging test takers to write about themes they might not have initially considered means the test is less dependent on prior knowledge of a given topic, and more focused on the ability to write effectively about a range of topics.
This interactivity goes hand-in-hand with the adaptive nature of the DET, in which test takers receive items based on their ability level. For example, if a test taker performs well on initial items, the test can present more challenging questions, ensuring that the test remains relevant and appropriately difficult for individuals of varying English proficiency levels.
How does Interactive Writing use AI?
This is the first time a test has used generative AI in real-time to create an interactive experience in an academic environment. Like other DET items, this task uses large language models (LLMs) for scaling item creation, adding to the enormous pool of items the test engine draws from to adaptively administer each test session.
For Interactive Writing, we not only use LLMs to generate a wide variety of writing prompts, we also use them to identify common themes that would likely come up in response to those prompts, and generate corresponding follow-up prompts to encourage test takers to expand on their original responses. This enables us to administer the item with responsive prompting: unlike on traditional writing assessments with static prompts, after a test taker writes an initial response, the model analyzes it in real time, and selects a customized follow-up prompt that asks the test taker to elaborate on a theme not addressed in their first response.
This mimics real-world academic writing, where writers often receive feedback and are asked to consider additional perspectives or deepen their analysis. Like in real-world academic/professional writing, writers in this task will have to consider different aspects of a topic and elaborate with more information or clarification on certain points, with examples and narratives, or by backing up their claims with concrete details.
Interactive tasks test students' true ability!
By closely mimicking real-world language use, interactive tasks offer a more relevant and engaging testing experience. Their active nature can heighten both motivation and focus, and potentially reducing test-related anxiety. As a result, test takers may perform better, leading to a more precise evaluation of a test taker’s true abilities, in a fraction of the time!