Automated Feedback for Students' Argumentative Essays
Get insights into labeling text data and prompt design using large language models.
Thorben Jansen, IPN Kiel Nils-Jonathan Schaller, IPN Kiel
We are a research project investigating how automated feedback and text assessment can promote students' written argumentation. The project aims to develop and evaluate a digital learning tool for students and teachers. Students will receive automated feedback on their writing and teachers will get an overview of the strengths and weaknesses in their students' written argumentations. We developed argumentation tasks on controversial, socially relevant, real-world problems informed by science (socio-scientific issues), which are situated in the context of climate change. As the basis of our machine learning model, we ask 1500 secondary school students to complete the tasks. Two specially trained raters will rate each argument's content and structure in every text in the resulting corpus.
100 annotated texts and annotation guidelines
We have two goals that we would like to work on with you: first, creating annotation guidelines and training annotators. Second, the partial automation of annotations through zero-shot classifications by use of prompt design.