Keywords
ChatGPT
Vocabulary Test Design
EFL/ESP Classrooms
AI-Generated Assessments
Language Testing Criteria
How to Cite
Abstract
This article explores the application of Artificial Intelligence (AI)-driven tools, particularly ChatGPT, for creating vocabulary test tasks in EFL/ESP classrooms. The research aims to evaluate the quality of vocabulary test tasks generated by ChatGPT by applying established criteria, including relevance, reliability, interactiveness, practicality, and impact. It investigates how ChatGPT-generated tasks meet these criteria and provides practical recommendations for educators to optimize the quality of AI-generated assessments. The authors indicate that criteria such as relevance, practicality, interactivity, and impact can be fully satisfied in ChatGPT-generated tests. However, the research identifies challenges with the reliability of AI-generated test tasks, primarily due to ambiguities in response choices.
The article emphasizes the pivotal role of human intervention in guiding and refining AI-generated outputs. Detailed and context-specific prompts crafted by educators are critical to maximizing the potential of ChatGPT while mitigating its limitations. To support EFL/ESP teachers, the study offers detailed recommendations for enhancing ChatGPT-generated test tasks, such as developing precise prompts, setting clear contexts, assigning specific roles to ChatGPT, and iteratively refining outputs. These strategies improve the reliability and effectiveness of AI-generated assessments and align them with pedagogical standards. The authors emphasise the importance of integrating human oversight with AI tools to maintain the validity and usefulness of language tests. This research contributes to the broader discourse on integrating AI in education by demonstrating how educators can leverage ChatGPT for test design while addressing its limitations. Future directions include evaluating the effectiveness of other types of AI-generated test tasks, exploring AI’s role in automated assessment and feedback, and examining the long-term impact of AI-driven assessments on teaching methodologies and students’ vocabulary acquisition in ESP contexts.
References
C. Zhai, S. Wibowo, and L. D. Li, “The effects of over-reliance on AI dialogue systems on students' cognitive abilities: a systematic review,” Smart Learn. Environ., vol. 11, p. 28, 2024. [Online]. Available: https://doi.org/10.1186/s40561-024-00316-7 (in English)
S. Akgun and C. Greenhow, “Artificial intelligence in education: Addressing ethical challenges in K-12 settings,” AI Ethics, vol. 2, pp. 431–440, 2022. [Online]. Available: https://doi.org/10.1007/s43681-021-00096-7 (in English)
I. Zaiarna, O. Zhyhadlo, and O. Dunaievska, “ChatGPT in Foreign Language Teaching and Assessment: Exploring EFL Instructors’ Experience,” ITLT, vol. 102, no. 4, pp. 176–191, Sep. 2024. doi: 10.33407/itlt.v102i4.5716. (in English)
O. Oluwafemi Ayotunde, D. I. Jamil, and N. Cavus, “The Impact of Artificial Intelligence in Foreign Language Learning Using Learning Management Systems: a Systematic Literature Review,” ITLT, vol. 95, no. 3, pp. 215–228, Jun. 2023. doi: 10.33407/itlt.v95i3.5233. (in English)
A. Kyrpa, O. Stepanenko, V. Zinchenko, T. Datsiuk, I. Karpan, and N. Tilniak, “Artificial Intelligence Tools in Teaching Social and Humanitarian Disciplines,” ITLT, vol. 100, no. 2, pp. 162–179, Apr. 2024. doi: 10.33407/itlt.v100i2.5563. (in English)
T. Schmidt and T. Strasser, “Artificial Intelligence in Foreign Language Learning and Teaching: A CALL for Intelligent Practice,” Anglistik: International Journal of English Studies, vol. 33, no. 1, pp. 165–184, Spring 2022. doi: 10.33675/ANGL/2022/1/14. (in English)
F. Karataş, F. Y. Abedi, F. O. Gunyel, et al., “Incorporating AI in foreign language education: An investigation into ChatGPT’s effect on foreign language learners,” Educ Inf Technol, vol. 29, pp. 19343–19366, 2024. [Online]. Available: https://doi.org/10.1007/s10639-024-12574-6 (in English)
H. Crompton, A. Edmett, and N. Ichaporia, “Artificial intelligence and English language teaching: A systematic literature review,” British Council, 2023. [Online]. Available: https://www.britishcouncil.org/sites/default/files/ai_in_english_language_teaching_systematic_review.pdf?utm_source=chatgpt.com (in English)
A. I. Mugableh, “The Impact of ChatGPT on the Development of Vocabulary Knowledge of Saudi EFL Students,” Arab World English Journal (AWEJ), Special Issue on ChatGPT, pp. 265–281, Apr. 2024. doi: https://dx.doi.org/10.24093/awej/ChatGPT.18. (in English)
K. K. Davis, “A New Parlor is Open: Legal Writing Faculty Must Develop Scholarship on Generative AI and Legal Writing,” Stetson Law Review Forum, vol. 7, no. 1, pp. 1, 2024. [Online]. Available: https://www2.stetson.edu/law-review/article/a-new-parlor-is-open-legal-writing-faculty-must-develop-schola. (in English)
H. Brown, Language Assessment: Principles and Classroom Practices, White Plains, NY: Longman, 2004, p. 33. (in English)
L. Bachman and A. Palmer, Language Testing in Practice: Designing and Developing Useful Language Tests, Oxford: Oxford University Press, 1996. (in English)
A. Hughes, Testing for Language Teachers, 2nd ed., Cambridge: Cambridge University Press, 2003.
How to write AI prompts that get results. [Online]. Available: https://blog.type.ai/post/how-to-write-ai-prompts-that-get-results. [Accessed: Dec. 30, 2024]. (in English)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright (c) 2025 Olena Zhyhadlo, Inna Zaiarna