USE OF FACIAL EMOTION RECOGNITION IN E-LEARNING SYSTEMS
Keywords:e-learning, facial emotion recognition, computer vision, machine learning, virtual classroom
AbstractSince the personal computer usage and internet bandwidth are increasing, e-learning systems are also widely spreading. Although e-learning has some advantages in terms of information accessibility, time and place flexibility compared to the formal learning, it does not provide enough face-to-face interactivity between an educator and learners. In this study, we are proposing a hybrid information system, which is combining computer vision and machine learning technologies for visual and interactive e-learning systems. The proposed information system detects emotional states of the learners and gives feedback to an educator about their instant and weighted emotional states based on facial expressions. In this way, the educator will be aware of the general emotional state of the virtual classroom and the system will create a formal learning-like interactive environment. Herein, several classification algorithms were applied to learn instant emotional state and the best accuracy rates were obtained using kNN and SVM algorithms.
A. Nagy, "The Impact of E-Learning", in: P.A. Bruck, A. Buchholz, Z. Karssen, A. Zerfass, (Eds). E-Content: Technologies and Perspectives for the European Market. Berlin: Springer-Verlag, pp. 79–96, 2005. (in English)
M. D. Lytras and A. Pouloudi, "E-Learning: Just a Waste Of Time," in Seventh Americas Conference on Information Systems, America, 2001. (in English)
S. Carliner and P. Shank, The e-Learning Handbook: Past Promises, Present Challenges, John Wiley & Sons, 2008. (in English)
A. Hicken, "2016 Elearning Hype Curve Predictions", 23.12.2015. [Online]. Available: http://www.webcourseworks.com/2016-elearning-hype-curve-predictions/. [Accessed 06.07.2017].
K. I. Benta, C. Marcel and F. M. Vaida, "A multimodal affective monitoring tool for mobile learning," Institute of electrical and electronics engineers computer society, pp. 34-38, 2015. (in English)
M. Lewis, H. Jones and F. Barrett, Handbook Of Emotions, New York, 2000. (in English)
A. M. Isen, "Positive Affect Influences Decision Making," in Handbook of emotions, M. Lewis and J. Haviland, Eds., Guilford, New York, The Guilford Press, p. 720, 2000. (in English)
N. Razon, "Okul Başarisini Etkileyen Faktörler", [Online]. Available: http://www.ekipnormarazon.com/makale-detay/okul-basarisini-etkileyen-faktorler
[Accessed 06.07.2017]. (in Turkish)
M. Feidakis, "A Review of Emotion-Aware Systems for e-Learning in Virtual Environments," in Formative Assessment, Learning Data Analytics and Gamification, S. Caballé and R. Clarisó, Eds., Academic Press, Boston, p.217-242, 2016. (in English)
W. K. Horton, Leading e-learning, American Society for Training and Development, 2001. (in English)
T. S. Ashwin, J. Jose, G. Raghu and G. R. Reddy, "An E-learning System With Multifacial Emotion Recognition Using Supervised Machine Learning," in IEEE Seventh International Conference on Technology for Education, 2015. (in English)
A. Al-Awni, "Mood Extraction Using Facial Features to ImproveLearning Curves of Students in E-Learning Systems," International Journal of Advanced Computer Science and Applications, vol. 7, no. 11, pp. 444-453, 2016. (in English)
L. B. Krithika and G. G. Lakshmi Priyya, "Student Emotion Recognition System (SERS) for e-learning," Procedia Computer Science, p. 767 – 776, 2016. (in English)
M. Magdin, M. Turcani and L. Hudec, "Evaluating the Emotional State of a User Using a Webcam," Special Issue on Artificial Intelligence Underpinning, vol. 4, no. 1, pp. 61-68, 2016. (in English)
C. H. Chu, W. J. Tsai, M. J. Liao and Y. M. Chen, "Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning," Soft Computing, pp. 1-27, 2017. (in English)
A. W. Wai, S. M. Tahir and Y. C. Chang, "GPU Acceleration of Real Time Viola-Jones Face Detection," in IEEE International Conference on Control System, Computing and Engineering, 2015. (in English)
P. A. Viola and M. J. Jones, "Rapid object detection using a boosted cascade," IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR, pp. 511-518, 2001. (in English)
P. Ekman and D. Keltner, "Universal Facial Expression of Emotion:An Old Controversy and New Findings," Nonverbal Behav,, vol. 21, no. 1, pp. 3-21, 1997. (in English)
C. Sagonas, G. Tzimiropoulos, S. Zafeiriou and M. Pantic, "300 Faces in-the-Wild Challenge: The first facial landmark localization Challenge," in The IEEE International Conference on Computer Vision, 2013. (in English)
P. N. Belhumeur, D. W. Jacobs, D. J. Kriegman and N. Kumar, "Localizing parts of faces using a consensus of exemplars," in The IEEE International Conference on Computer Vision and Pattern Recognation (CVPR), 2011. (in English)
V. Le, J. Brandt, L. Bourdev and T. S. Huang, "Interactive Facial Feature Localization," in European Conference on Computer Vision, 2012. (in English)
X. Zhu and D. Ramanan, "Face detection, pose estimation, and landmark localization in the wild," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012. (in English)
M. Kostinger, P. Wohlhart, P. M. Roth and H. Bischof, "Annotated facial landmarks in the wild: A large-scale, realworld database for facial landmark localization," in IEEE International Conference on ComputerVision Workshops (ICCV Workshops), 2011. (in English)
U. Ayvaz and H. Gürüler, "The Detection of Emotional Expression towards Computer Users," International Journal of Informatics Technologies, vol. 10, no. 2, pp. 231-239, 2017. (in Turkish)
V. N. Vapnik, "An overview of statistical learning theory," IEEE transactions on neural networks , vol. 10, no. 5, pp. 988-999, 1999. (in English)
X. Wu, V. Kumar, J. R. Quinlan, J. Ghosh, Q. Yang and H. Motoda, "Top 10 algorithms in data mining," Knowledge and information systems, vol. 14, no. 1, pp. 1-37, 2008. (in English)
A. Liaw and M. Wiener, "Classification and regression by random Forest," R news, vol. 2, no. 3, pp. 18-22, 2002. (in English)
D. Steinberg and P. Colla, "CART: classification and regression trees," in The Top Ten Algorithms in Data Mining, X. Wu and V. Kumar, Eds., Chapman & Hall/CRC, 2009, p. 179. (in English)
How to Cite
Authors who publish in this journal agree to the following terms:
- Authors hold copyright immediately after publication of their works and retain publishing rights without any restrictions.
- The copyright commencement date complies the publication date of the issue, where the article is included in.
- Authors grant the journal a right of the first publication of the work under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License (CC BY-NC-SA 4.0) that allows others freely to read, download, copy and print submissions, search content and link to published articles, disseminate their full text and use them for any legitimate non-commercial purposes (i.e. educational or scientific) with the mandatory reference to the article’s authors and initial publication in this journal.
- Original published articles cannot be used by users (exept authors) for commercial purposes or distributed by third-party intermediary organizations for a fee.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) during the editorial process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (see this journal’s registered deposit policy at Sherpa/Romeo directory).
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Post-print (post-refereeing manuscript version) and publisher's PDF-version self-archiving is allowed.
- Archiving the pre-print (pre-refereeing manuscript version) not allowed.