A wave of new research is shedding light on artificial intelligence’s (AI) potential to change how we teach, learn, and assess student performance. From automating tedious grading tasks to enhancing lecture engagement, providing personalized feedback, and supporting the development of essential skills, AI is poised to become an indispensable tool for educators. Additional research also illuminates the risk of perpetuating biases and stereotypes.
Grading Assistant
The vast majority of research around technology and assessments focuses on the automated scoring of student work collected mostly through computers. However, a recent study, “Can AI Assistance Aid in the Grading of Handwritten Answer Sheets?” investigated the effectiveness of AI-assisted grading for handwritten responses. The researchers developed a system that automatically detects question regions, extracts student information, and highlights keywords to assist graders. The AI-assisted keyword highlighting feature led to a 31% reduction in average grading time per individual response and a 33% reduction for entire answer sheets. However, the system’s effectiveness varied depending on the question type, with short answer and numerical questions benefiting more than long answer questions or those requiring evaluation of logic. That said, the study does point to how AI could be a helpful assistant to educators and others assessing student work.
Engaging Lectures: AI’s Role in Enhancing Teaching Effectiveness
Engaging students in the learning process is a critical aspect of effective teaching. The study “Is the Lecture Engaging for Learning” introduces an intelligent lecturing assistant system that analyzes lecture voice sentiment to enhance student engagement. The researchers developed a dataset of labeled lecture voice clips and evaluated various classification models to identify engaging and non-engaging lectures. The best-performing model demonstrated a remarkable ability to accurately identify boring lectures, correctly classifying 90% of the cases. This new approach to using AI could be combined with content analysis, pedagogical principles, and real-time interventions to improve teaching effectiveness and student engagement.
Multi-Level Feedback Generation: Empowering Novice Peer Counselors
Elsewhere, the study “Multi-Level Feedback Generation with Large Language Models for Empowering Novice Peer Counselors” showcases the development of a multi-level feedback framework for training peer counselors. The researchers created a dataset called FeedbackESConv by annotating 400 emotional support conversations with comprehensive feedback using a model-in-the-loop approach involving GPT-4 and domain experts. Experts agreed that the generated feedback would be helpful in training novice peer counselors in 90% of conversations. This study highlights the potential of AI to support the training of peer counselors, but one could also imagine this approach used for other training in areas as diverse as education, healthcare, and customer service.
Gender Stereotypes in STEM Education
A recent study evaluating ChatGPT revealed concerning gender biases in its career suggestions for children. When asked what boys and girls could become when they grow up, ChatGPT suggested STEM careers significantly more often for boys than girls, with the bias increasing with age. The study, conducted across English, Danish, Catalan, and Hindi speaking demographics, also found gender stereotypes in other career suggestions: girls received more suggestions related to arts, writing, animal care, and teaching, while boys had more related to architecture, sports, and finance. These findings raise concerns about the potential of AI language models like ChatGPT to reinforce harmful gender stereotypes in children at an age when they are making important educational choices. Applications providing this kind of education and career navigation will need robust red-teaming and fine-tuning to ensure better-aligned outputs.
Cognitive Biases in Problem Solving
The study “Do Language Models Exhibit the Same Cognitive Biases in Problem-Solving as Human Learners?” investigates whether LLMs exhibit cognitive biases akin to those observed in children when solving arithmetic word problems, particularly in the domains of text comprehension and solution planning. Mirroring the tendencies displayed by children, LLMs demonstrated improved performance on problems featuring consistent relational keywords compared to inconsistent ones (consistency bias) and on problems involving dynamic change of state versus static comparisons (transfer vs comparison bias). Intriguingly, however, LLMs diverged from children in the solution execution stage, showing no evidence of the “carry effect,” which typically manifests as decreased performance on arithmetic problems necessitating a carry operation. These findings suggest that while LLMs effectively simulate child-like biases in the initial stages of problem-solving, they employ fundamentally different mechanisms during the actual computation phase.
These studies offer a glimpse into some of the potential benefits and challenges of AI, streamlining administrative tasks and personalizing learning experiences to providing targeted feedback and supporting the development of essential skills. They contribute to our understanding but also underscore the need for more robust testing, fine-tuning of the models, and additional research to understand how these models can be better aligned.
The post New Research Contributes to Our Understanding of AI appeared first on American Enterprise Institute – AEI.