Applications of Large Multimodal Models (LMMs) in STEM Education: From Visual Explanations to Virtual Experiments

Abstract

Large Multimodal Models (LMMs) represent a paradigm shift in artificial intelligence, extending beyond the text-centric capabilities of their predecessors to process, interpret, and generate information across diverse modalities including images, video, audio, and code. This paper provides a comprehensive review of the applications of LMMs in Science, Technology, Engineering, and Mathematics (STEM) education, with a particular focus on their potential to revolutionize visual explanations and virtual experiments. It examines how LMMs can generate dynamic and interactive scientific visualizations, make abstract concepts more tangible, and power sophisticated virtual laboratories that foster inquiry-based learning and problem-solving skills. The paper delves into relevant learning theories, such as constructivism and Cognitive Load Theory, and explores pedagogical frameworks like Inquiry-Based Learning and the ARCHED model, adapting them for LMM integration. Critical challenges, including ensuring scientific accuracy, mitigating cognitive biases, upholding academic integrity, and managing cognitive load, are discussed in depth. Furthermore, the paper analyzes emerging assessment strategies, such as the FACT and HCAIF frameworks, that account for human-AI collaboration and the importance of student metacognition. Ethical considerations surrounding data privacy, algorithmic bias, and equitable access are also addressed. Finally, the paper outlines future trajectories, including the need for teacher professional development, the design of next-generation AI-driven learning environments, the evolving nature of the scientific method in an era of human-AI partnership, and key research priorities. The overarching argument is that while LMMs offer transformative potential for STEM education, their successful and ethical integration requires careful pedagogical planning, critical evaluation, and collaborative research among educators, developers, and policymakers.