Kinect camera sensor-based pointing motion recognition with magnification function to increase concentration among children during remote classes

Proceedings of ‏The 4th International Conference on Advanced Research in Education, Teaching and Learning

Year: 2021

DOI: https://www.doi.org/10.33422/4th.aretl.2021.05.61

[Fulltext PDF]

Kinect camera sensor-based pointing motion recognition with magnification function to increase concentration among children during remote classes

Von Ralph Dane Marquez Herbuela, Yeonkyung Eom, Tomonori Karita, Yoshinori Wada, Satoru Shibata, Shenglin Mu, Takaya Omori

 

ABSTRACT: 

The sudden shift of schools to distance learning due to the COVID-19 pandemic has also impacted hospitalized children with severe medical, physical, or neurological conditions. We developed a system that enables children to attend classes using an alter-ego robot that lets them virtually experience an actual classroom environment. However, due to the inability of the existing camera and monitors to automatically adjust and magnify pivotal information that teachers present, another challenge is to draw children’s attention and increase or maintain their concentration and focus during remote classes. Thus, we proposed and developed a system that automatically magnifies the view around where a pointing motion is detected. In this study, we described the design of our proposed camera-sensor-based pointing motion recognition with magnification function system using Kinect which consists of an RGB (color) camera, an IR (infrared) camera, an emitter-based Depth sensor, and four microphone arrays. We also conducted multiple experiments to extract pointing motions and evaluate the ability of the system to detect them using different parameters such as magnification rate, change-of-position tracking rate, and reset-position rate which were consequently used for Semantic Differential analysis. The optimal conditions and algorithms (e.g. upper body limb positioning, camera-blackboard-body distance and height measurements, color dimensions, user-monitor distance, etc.) were also investigated for system stability.

Keywords: Remote learning support system; Kinect sensor; Pointing motion; semantic differential method.