Real time music transcription [closed]

Question

I'm trying to implement a system that can produce a sheet of music for a piano piece as and when it is being played, that is, it should be a real time transcription and not that of a recorded piece.

So far I've been testing on wav files, using FFt on MATLAB, but when it comes to real time, the system needs to be really-really fast.

Any suggestions on what kind of approach I could use, the type of board (I've thought of using Arduino). I could use, because I'd like to directly transfer my MATLAB code rather than converting it to C.

Answer 1

Any suggestions on what kind of approach I could use, the type of board (I've thought of using Arduino). I could use, because I'd like to directly transfer my MATLAB code rather than converting it to C.

You can have different approaches to your project that I can think of:

1. First there are some pianos that have an electronic device that can output the played notes through a midi interface, with velocity and position. It's the most accurate and fast solution to track played notes, though not much DIY. On the computer side, you'll only need a midi interface (that can be easily done using an arduino) and have a software that transforms MIDI notes into a score ;
2. Then, you can mimic that behavior, by wiring the keyboard of the piano to sensors that will be able to measure velocity and which note has been played. There are many strategies you can use (from using lasers on the mechanics, to pressure switches...), it all depends on the investment you're ready to make ; There an arduino (like the mega or the due) can be a good solution.
3. But as you were talking about FFTs, I assume you want to record the sound made by the piano and detect keys. There you hit state of the art knowledge on music research, the problem you'll meet is the multiple f0 detection in polyphonic music, though there are strategies specific to piano, it's still a very active research field.
4. Let's assume you're only doing monophonic detection for the transcription. The problem will be that the Analogical-Digital-Converter included in the Arduino just sucks, it has a definition of 10bit which is way too low to detect anything useful. With the Arduino DUE, you may have a bit more definition as it features a 16-channel 12-bit 1Msps ADC. Which is still not much for a good transcription. So you'll either have to create use an external ADC like the Linear LTC2383-16 which is 16bit 1Msps and has a SPI interface, the minimum to begin playing. Behind that you'll need to find the correct microphone and create the good electronics between the ADC and the microphone (which I don't know much about).
5. Finally, would'nt it be easier to take a small computer (like the Beaglebone or the RaspberryPI5), plug in a really good USB ADC/sound card, and use that board to pipe through the transcription? You may as well run your software on a cloud server (like google appengine or AWS) that has enough processor to make your Matlab run happily.

So to make a good real time and embedded transcription of music, the best solution is the first solution (or second if you really want to go DIY). If you really want to go through the "air" interfaces, I think the good solution is the fifth.

And finally, about the code, I'd advice you to convert your code from Matlab to Python (if you don't want to write C/C++ code) it will run significantly faster. So you can figure out how bad Matlab is, when I was working for the IRCAM, I have written a tool that was converting Matlab source code to C++ source code, and once compiled result were 40x faster.

HTH