Keywords
Community of Inquiry
Machine Learning
Natural Language Processing
Classification
How to Cite
Abstract
The Community of Inquiry (CoI) framework has been widely employed for the past two decades to assess the knowledge gained and shared through online discussion forums. The cognitive presence component of the CoI framework helps identify the evidence of thoughtful knowledge reconstructions through meaning-making during inquiry-based learning. Identifying and scoring these cognitive presences is essential for assessing the students’ learning achievements through online discussion forums. Considering the difficulties associated with manual coding and identifying cognitive presences in discussion forums and the limitations in the existing techniques for auto-identifying and scoring cognitive presences, this research attempted to develop a more efficient tool to identify and score cognitive presences in online discussion forums. The research employed the constructive research approach. The methodology integrated Random Forest (RF) classification with TF-IDF feature extraction and Support Vector Machine (SVM) classification with Word2Vec embedding. A rule-based classifier, constructed upon indicator mappings, enriched the classification process. A weighted voting ensemble method was employed to combine the outputs of the individual classifiers. Our approach was trained and tested on two datasets comprising 781 messages containing 47,592 words. This ensemble method demonstrated notable efficacy, achieving a 69% accuracy rate in classification tasks. This highlights the robustness of the combined approach in enhancing classification performance. Furthermore, the study introduces a scoring model that calculates individual student scores based on post categories, enabling detailed evaluations of student engagement and participation. By assigning scores reflective of discussion contributions, this model advances comprehensive assessments of online learning interactions. Our work contributes to the ongoing conversation on leveraging machine learning for cognitive analysis in online learning environments, highlighting the importance of context-specific methodologies in advancing educational assessment practices.
References
J. Peral, A. Ferrández, H. Mora, D. Gil and E. Kauffmann, "A Review of the Analytics Techniques for an Efficient Management of Online Forums: An Architecture Proposal," IEEE, vol. 7, pp. 12220-12240, 2019. (in English)
B. P. Nunes, A. Mera, R. Kawase, B. Fetahu, M. A. Casanova and G. H. B. d. Campos, "A topic extraction process for online forums," in 2014 IEEE 14th International Conference on Advanced Learning Technologies, 2014. (in English)
D. Distante, A. Fernandez, L. Cerulo and A. Visaggio, "Enhancing Online Discussion Forums with Topic-Driven Content Search and Assisted Posting," in International Joint Conference on Knowledge Discovery, Knowledge Engineering, and Knowledge Management, Rome, Italy, 2015. (in English)
N. A. Albatayneh, K. I. Ghauth and F.-F. Chua, "Utilizing Learners’ Negative Ratings in Semantic Content-based Recommender System for e-Learning Forum," Journal of Educational Technology & Society, vol. 21, no. 1, p. 112–125, 2018. (in English)
G. Gorgun, S. N. Yildirim-Erbasli, and C. D. Epp, "Predicting Cognitive Engagement in Online Course Discussion Forums," в International Educational Data Mining Society, Durham, United Kingdom, 2022. (in English)
T. A. Weerasinghe, K. P. Hewagamage and R. Ramberg, "Re-evaluation of community of inquiry model," The International Journal on Advances in ICT for Emerging Regions, vol. 5, no. 4, pp. 1-13, 2012. (in English)
D. R. Garrison, T. Anderson and W. Archer, "Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education," The Internet and Higher Education, vol. 2, no. 2-3, p. 87–105, 1999. (in English)
D. H.-C. Cheung, A. K.-L. Ng, K.-M. Kiang and H. H.-Y. Chan, "Creating a community of inquiry in the science classroom: an effective pedagogy for teaching diverse students?," Journal of Further and Higher Education, vol. 44, no. 1, pp. 1-13, 2020. (in English)
D. c. K. Kozan and J. C. Richardson, "Interrelationships between and among social, teaching, and cognitive presence," The Internet and Higher Education, vol. 21, pp. 68-73, 2014. (in English)
P. Shea, J. Richardson and K. Swan, "Building bridges to advance the Community of Inquiry framework for online learning," Educational Psychologist, vol. 57, no. 3, pp. 148-161, 2022. (in English)
D. R. Garrison, T. Anderson and W. Archer, "Critical thinking, cognitive presence, and computer conferencing in distance education," American Journal of Distance Education, vol. 15, no. 1, pp. 7-23, 2001. (in English)
Y.-L. Lee, "Co-Creating Educational YouTube Videos as Site as a Community of Inquiry," Scholarly and Research Communication, vol. 13, no. 2, pp. 15-pp, 2022. (in English)
Y. Taniguchi, S. Konomi and Y. Goda, "Examining Language-Agnostic Methods of Automatic Coding in the Community of Inquiry Framework," in International Association for Development of the Information Society, Cagliari, Italy, 2019. (in English)
K. F. Hew, X. Hu, C. Qiao and Y. Tang, "What predicts student satisfaction with MOOCs: A gradient boosting trees supervised machine learning and sentiment analysis approach," Computers & Education, vol. 145, p. 103724, 2020. (in English)
V. Kovanović, S. Joksimović, Z. Waters, D. Gašević, K. Kitto, M. Hatala and G. Siemens, "Towards automated content analysis of discussion transcripts: a cognitive presence case," Proceedings of the Sixth International Conference on Learning Analytics & Knowledge, pp. 15-24, 2016. (in English)
V. Neto, V. Rolim, R. Ferreira, V. Kovanović, D. Gašević, R. D. Lins and R. Lins, "Automated Analysis of Cognitive Presence in Online Discussions Written in Portuguese," in European Conference on Technology Enhanced Learning, 2018. (in English)
V. Dornauer, M. Netzer, É. Kaczkó, L.-M. Norz and E. Ammenwerth, "Automatic Classification of Online Discussions and Other Learning Traces to Detect Cognitive Presence," International Journal of Artificial Intelligence in Education, vol. 34, p. 395–415, 2023. (in English)
V. Neto, V. Rolim, A. Pinheiro, R. D. Lins, D. Gaševic and R. F. Mello, "Automatic Content Analysis of Online Discussions for Cognitive Presence: A Study of the Generalizability Across Educational Contexts," IEEE Transactions on Learning Technologies, vol. 14, no. 3, pp. 299-312, 2021. (in English)
D. Gašević, O. Adesope, S. Joksimović and V. Kovanović, "Externally-facilitated regulation scaffolding and role assignment to develop cognitive presence in asynchronous online discussions," The Internet and Higher Education, vol. 24, pp. 53-65, 2015. (in English)
Y. Hu, C. Donald and N. Giacaman, "Cross Validating a Rubric for Automatic Classification of Cognitive Presence in MOOC Discussions," International Review of Research in Open and Distributed Learning, vol. 23, no. 2, pp. 242–260, 2022. (in English)
R. Al-Shabandar, A. J. Hussain, P. Liatsis and R. Keight, "Detecting At-Risk Students With Early Interventions Using Machine Learning Techniques," IEEE Access , vol. 7, pp. 149464 - 149478, 2019. (in English)
P. Leitner, M. Khalil та M. Ebner, «Learning Analytics in Higher Education—A Literature Review,» Learning Analytics: Fundaments, Applications, and Trends, p. 1–23, 2017. (in English)
B. Chen, Y.-H. Chang, F. Ouyang and W. Zhou, "Fostering student engagement in online discussion through social learning analytics," The Internet and Higher Education, vol. 37, pp. 21-30, 2018. (in English)
T. E. McKlin, Analyzing cognitive presence in online courses using an artificial neural network, Georgia State University, 2004. (in English)
S. Corich, K. Hunt and L. Hunt, "Computerised Content Analysis for Measuring Critical Thinking within Discussion Forums," Journal of e-Learning and Knowledge Society, vol. 2, no. 1, 2006.
V. Kovanovic, S. Joksimovic, D. Gasevic and M. Hatala, "Automated cognitive presence detection in online discussion transcripts," CEUR Workshop Proceedings, vol. 1137, 2014. (in English)
Z. Water, V. Kovanović, K. Kitto and D. Gašević, "Structure Matters: Adoption of Structured Classification Approach in the Context of Cognitive Presence Classification," in Information Retrieval Technology: 11th Asia Information Retrieval Societies Conference, AIRS 2015, Brisbane, QLD, Australia, 2016. (in English)
E. Farrow, J. Moore and D. Gašević, "Analysing discussion forum data: a replication study avoiding data contamination," in Proceedings of the 9th International Conference on Learning Analytics & Knowledge, 2019.
G. Barbosa, R. Camelo, A. P. Cavalcanti, P. Miranda, R. F. Mello, V. Kovanović and D. Gašević, "Towards automatic cross-language classification of cognitive presence in online discussions," Proceedings of the Tenth International Conference on Learning Analytics & Knowledge, pp. 605-614, 2020. (in English)
H. Hayati, A. Chanaa, M. K. Idrissi and S. Bennani, "Doc2Vec & Naïve Bayes: Learners’ Cognitive Presence Assessment through Asynchronous Online Discussion TQ Transcripts," International Journal of Emerging Technologies in Learning, vol. 14, no. 8, 2019. (in English)
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.
Copyright (c) 2025 P.A.L. Nadeesha, T.A. Weerasinghe, W.R.N.S Abeyweera