USE OF FACIAL EMOTION RECOGNITION IN E-LEARNING SYSTEMS

  • Uğur Ayvaz Mugla Sitki Kocman University, Mugla
  • Hüseyin Gürüler Mugla Sitki Kocman University, Mugla
  • Mehmet Osman Devrim Mugla Sitki Kocman University, Mugla
Keywords: e-learning, facial emotion recognition, computer vision, machine learning, virtual classroom

Abstract

Since the personal computer usage and internet bandwidth are increasing, e-learning systems are also widely spreading. Although e-learning has some advantages in terms of information accessibility, time and place flexibility compared to the formal learning, it does not provide enough face-to-face interactivity between an educator and learners. In this study, we are proposing a hybrid information system, which is combining computer vision and machine learning technologies for visual and interactive e-learning systems. The proposed information system detects emotional states of the learners and gives feedback to an educator about their instant and weighted emotional states based on facial expressions. In this way, the educator will be aware of the general emotional state of the virtual classroom and the system will create a formal learning-like interactive environment. Herein, several classification algorithms were applied to learn instant emotional state and the best accuracy rates were obtained using kNN and SVM algorithms.

Author Biographies

Uğur Ayvaz, Mugla Sitki Kocman University, Mugla
MSc in Information Systems Engineering (ongoing), Department of Information Systems Engineering, Research Assistant
Hüseyin Gürüler, Mugla Sitki Kocman University, Mugla
PhD in Electronic and Computer Education, Department of Information Systems Engineering, Assistant Professor
Mehmet Osman Devrim, Mugla Sitki Kocman University, Mugla
MSc in Information Systems Engineering (ongoing), Department of Information Systems Engineering, Student

References

A. Nagy, "The Impact of E-Learning", in: P.A. Bruck, A. Buchholz, Z. Karssen, A. Zerfass, (Eds). E-Content: Technologies and Perspectives for the European Market. Berlin: Springer-Verlag, pp. 79–96, 2005. (in English)

M. D. Lytras and A. Pouloudi, "E-Learning: Just a Waste Of Time," in Seventh Americas Conference on Information Systems, America, 2001. (in English)

S. Carliner and P. Shank, The e-Learning Handbook: Past Promises, Present Challenges, John Wiley & Sons, 2008. (in English)

A. Hicken, "2016 Elearning Hype Curve Predictions", 23.12.2015. [Online]. Available: http://www.webcourseworks.com/2016-elearning-hype-curve-predictions/. [Accessed 06.07.2017].

(in English)

K. I. Benta, C. Marcel and F. M. Vaida, "A multimodal affective monitoring tool for mobile learning," Institute of electrical and electronics engineers computer society, pp. 34-38, 2015. (in English)

M. Lewis, H. Jones and F. Barrett, Handbook Of Emotions, New York, 2000. (in English)

A. M. Isen, "Positive Affect Influences Decision Making," in Handbook of emotions, M. Lewis and J. Haviland, Eds., Guilford, New York, The Guilford Press, p. 720, 2000. (in English)

N. Razon, "Okul Başarisini Etkileyen Faktörler", [Online]. Available: http://www.ekipnormarazon.com/makale-detay/okul-basarisini-etkileyen-faktorler

[Accessed 06.07.2017]. (in Turkish)

M. Feidakis, "A Review of Emotion-Aware Systems for e-Learning in Virtual Environments," in Formative Assessment, Learning Data Analytics and Gamification, S. Caballé and R. Clarisó, Eds., Academic Press, Boston, p.217-242, 2016. (in English)

W. K. Horton, Leading e-learning, American Society for Training and Development, 2001. (in English)

T. S. Ashwin, J. Jose, G. Raghu and G. R. Reddy, "An E-learning System With Multifacial Emotion Recognition Using Supervised Machine Learning," in IEEE Seventh International Conference on Technology for Education, 2015. (in English)

A. Al-Awni, "Mood Extraction Using Facial Features to ImproveLearning Curves of Students in E-Learning Systems," International Journal of Advanced Computer Science and Applications, vol. 7, no. 11, pp. 444-453, 2016. (in English)

L. B. Krithika and G. G. Lakshmi Priyya, "Student Emotion Recognition System (SERS) for e-learning," Procedia Computer Science, p. 767 – 776, 2016. (in English)

M. Magdin, M. Turcani and L. Hudec, "Evaluating the Emotional State of a User Using a Webcam," Special Issue on Artificial Intelligence Underpinning, vol. 4, no. 1, pp. 61-68, 2016. (in English)

C. H. Chu, W. J. Tsai, M. J. Liao and Y. M. Chen, "Facial emotion recognition with transition detection for students with high-functioning autism in adaptive e-learning," Soft Computing, pp. 1-27, 2017. (in English)

A. W. Wai, S. M. Tahir and Y. C. Chang, "GPU Acceleration of Real Time Viola-Jones Face Detection," in IEEE International Conference on Control System, Computing and Engineering, 2015. (in English)

P. A. Viola and M. J. Jones, "Rapid object detection using a boosted cascade," IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR, pp. 511-518, 2001. (in English)

P. Ekman and D. Keltner, "Universal Facial Expression of Emotion:An Old Controversy and New Findings," Nonverbal Behav,, vol. 21, no. 1, pp. 3-21, 1997. (in English)

C. Sagonas, G. Tzimiropoulos, S. Zafeiriou and M. Pantic, "300 Faces in-the-Wild Challenge: The first facial landmark localization Challenge," in The IEEE International Conference on Computer Vision, 2013. (in English)

P. N. Belhumeur, D. W. Jacobs, D. J. Kriegman and N. Kumar, "Localizing parts of faces using a consensus of exemplars," in The IEEE International Conference on Computer Vision and Pattern Recognation (CVPR), 2011. (in English)

V. Le, J. Brandt, L. Bourdev and T. S. Huang, "Interactive Facial Feature Localization," in European Conference on Computer Vision, 2012. (in English)

X. Zhu and D. Ramanan, "Face detection, pose estimation, and landmark localization in the wild," in IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2012. (in English)

M. Kostinger, P. Wohlhart, P. M. Roth and H. Bischof, "Annotated facial landmarks in the wild: A large-scale, realworld database for facial landmark localization," in IEEE International Conference on ComputerVision Workshops (ICCV Workshops), 2011. (in English)

U. Ayvaz and H. Gürüler, "The Detection of Emotional Expression towards Computer Users," International Journal of Informatics Technologies, vol. 10, no. 2, pp. 231-239, 2017. (in Turkish)

V. N. Vapnik, "An overview of statistical learning theory," IEEE transactions on neural networks , vol. 10, no. 5, pp. 988-999, 1999. (in English)

X. Wu, V. Kumar, J. R. Quinlan, J. Ghosh, Q. Yang and H. Motoda, "Top 10 algorithms in data mining," Knowledge and information systems, vol. 14, no. 1, pp. 1-37, 2008. (in English)

A. Liaw and M. Wiener, "Classification and regression by random Forest," R news, vol. 2, no. 3, pp. 18-22, 2002. (in English)

D. Steinberg and P. Colla, "CART: classification and regression trees," in The Top Ten Algorithms in Data Mining, X. Wu and V. Kumar, Eds., Chapman & Hall/CRC, 2009, p. 179. (in English)

Published
2017-09-30
Section
ICT and learning tools in the higher education establishments