- The overall goal of the VirtualPianoLesson project is to create a mixed reality piano lesson, that shows the student a visual representation of the musical piece they want to learn, tracks the progress during the virtual practice session, and adapts the difficulty level and visual representation of the lesson to the student’s ability.
- This project/thesis focusses on the development of mixed reality piano playing.
- During the project, different parts of the mixed reality environment need to be synchronized (real/virtual keyboard, user vision, user’s hands).
- The mixed reality environment will be implemented with a HTC Vive using Unity.
- How can we synchronize a virtual and real keyboard using a HTC Vive?
- How can we trace the user's vision and hand movement on the keyboard using a HTC Vive?
- Desnoyers-Stewart, J., Gerhard, D., & Smith, M. (2017). Mixed Reality MIDI Keyboard. In Proceedings of the 13th International Symposium on CMMR (pp. 376-386).
- Desnoyers-Stewart, J., Gerhard, D., & Smith, M. L. (2018). Augmenting a MIDI Keyboard Using Virtual Interfaces. Journal of the Audio Engineering Society, 66(6), 439-447.
- Liang, Hui, et al. "Barehanded music: real-time hand interaction for virtual piano." Proceedings of the 20th ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games. ACM, 2016