Towards an end-to-end music transcription system using neural networks

Automatic music transcription;Machine learning;Deep learning;Signal processing

Correa Carvalho, Ralf Gunter ; Smaragdis ; Paris

关键词: Automatic music transcription;    Machine learning;    Deep learning;    Signal processing;   

Others  :  https://www.ideals.illinois.edu/bitstream/handle/2142/99126/CORREACARVALHO-THESIS-2017.pdf?sequence=1&isAllowed=y

美国|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

Transcription is the task of writing down instructions on how to play a particular piece of music, including individual notes, note durations, embellishments and so on. While most major works in the traditional repertoire have readily available transcriptions for various instrument arrangements, this is not as common in genres where improvisation is more prevalent, such as Jazz, or where the piece has a very particular purpose, as in motion picture and video game soundtracks. It has notable parallels with the task of Automatic Speech Recognition (ASR) and indeed from this connection arises some natural Machine Learning-based approaches. However, these methods usually involve carefully designed preprocessing steps, or transcription into less flexible representations, such as piano rolls, which are harder to read for humans. This work investigates the feasibility of designing an end-to-end music transcription system that takes in raw audio recordings and produces Lilypond notation, which can directly generate easily-recognizable sheet music. In keeping with modern ASR methods, this task is modeled as a sequence-to-sequence problem using Convolutional and Recurrent Neural Networks. The system is shown to perform well for both monophonic (single melody on a single instrument) and polyphonic music (parallel melodies on possibly different instruments) for randomly generated pieces played by the piano and various other common orchestra instruments.