Standardized Assessment of Artificial Intelligence Literacy: Development and Validation of the Multidimensional AI Literacy Competency Scale (MAIL-CS)

Abstract

Generative AI’s rapid diffusion demands precise, up-to-date measures of AI literacy. This study develops and validates the Multidimensional AI Literacy Competency Scale (MAIL-CS), designed specifically for the GenAI era. Using a large sample of Chinese university students (N=850) and a split-sample design, we conducted EFA and CFA to establish a robust four-factor structure—Foundational Knowledge & Ethics, Operational Skills, Critical Evaluation, and Application & Innovation. The best-fitting model showed strong fit indices, and the 32-item scale demonstrated high internal consistency (Cronbach’s α and McDonald’s ω ≥ .82 subscales; α=.91, ω=.92 total). Convergent validity was supported by positive correlations with digital literacy and critical thinking; discriminant validity was evidenced by negligible relations with Big Five traits. MAIL-CS offers educators, researchers, and policymakers a reliable instrument to diagnose competency gaps, evaluate interventions, and inform curriculum and strategy. Validation in a non-Western context provides a foundation for cross-cultural assessment and future invariance testing.