Gesture Recognition & Chanting Assessment For Byzantine Music Learning

Proceedings of The 2nd International Conference on Advanced Research in Education

Year: 2019


[Fulltex PDF]

Gesture Recognition & Chanting Assessment For Byzantine Music Learning

Kostas Kokkinidis, Theodoros Mastoras, Athanasia Stergiaki and Paraskevi Kritopoulou



Recent works related to digital self-instruction environments, present scarce efforts to provide combined instruction for gestural and vocal skills. Based upon a recently introduced learning and teaching method of vocal music, this research utilizes existing technologies to achieve the development of such a learning environment. The presented system administers the learning experience in order to improve the motion, sound and rhythm related skills of the student. Student performance is compared with a pre-recorded instructor performance in order to provide customized feedback that bespeaks the flaws of the former performance. Motion and sound-capturing technologies are combined, and related feature extraction algorithms are applied. The gestural and vocal features of the instructor performance are compared off-line to those of the student performance, in order to detect the differences, while the tempo is indicated through gestures. The system evaluates constantly the performances in order to provide visual feedback based on their differences. The aim is for the student to reproduce the instructor performance in an approximate manner. An assessment formula for the student performance is proposed and tested for its validity and accuracy. The selected musical genre on which this system was applied is Byzantine music, since its complexity and variety tests the existing sound recognition algorithms.

keywords: sensory-motor learning; Byzantine music; multimodal interaction; gesture recognition; singing voice assessment.