VirtualPianoLesson (3): Intelligent System for Tailored Real-Time Feedback during Piano Lessons

Theoretical (Analytical):

Practical (Implementation):

Literature Work:


  • The overall goal of the VirtualPianoLesson project is to create a mixed reality piano lesson, that shows the student a visual representation of the musical piece they want to learn, tracks the progress during the virtual practice session, and adapts the difficulty level and visual representation of the lesson to the student's ability.
  • This project/thesis focuses on the development of an intelligent feedback system, that evaluates the student's ability during piano practice.
  • During the project, different algorithms for ability detection and selection of fitting lectures will be compared.
  • The algorithms should be implemented in python and are using the keyboard sensors to intelligently select the appropriate visualization levels and next music pieces.

Problem Statement

  • How can we automatically derive ability levels and select appropriate visualization methods for musical pieces?
  • How can we predict fitting next lectures based on current performance during virtual piano lessons?


  • Assess existing pedagogical structures for piano lessons
  • Derive ability level and performance on current piece from keyboard sensors
  • Select fitting visual metaphors and lectures based on detected performance and ability


  • Interest in music/piano education
  • Knowledge about machine learning
  • Good programming skills in Python


    • Scope: Bachelor / Master
    • Project/Thesis Duration: 3 months/3 months (Bachelor), 6 months / 6 months (Master)
    • Start: immediately



    • Xiao Xiao, Pablo Puentes, Edith Ackermann, and Hiroshi Ishii. 2016. Andantino: Teaching Children Piano with Projected Animated Characters. In Proceedings of the The 15th International Conference on Interaction Design and Children (IDC '16). ACM, New York, NY, USA, 37-45. DOI:
    • Fukayama, Hiromi Nakamura Tomoyasu Nakano Satoru, and Masataka Goto. "ChordScanner: Browsing Chord Progressions based on Musical Typicality and Intra-Composer Consistency." (2018).
    • Jacobson, Jeanine M., E. L. Lancaster, and Albert Mendoza. Professional Piano Teaching, Volume 1-Elementary Levels: A Comprehensive Piano Pedagogy Textbook. Alfred Music, 2015.