How can a chromagram file produced by librosa be interpreted as a set of musical keys?

Question

I have a chroma features file here. How can these numbers be interpreted as belonging to different musical keys? I need to use the key found at a particular time code to produce a solution similar to this in order to mix between two tracks. How can these numbers be interpreted as an overall key being played and how can I skip to a particular time code to get the given key?

I've tried getting the chroma as described here, but the output is just in numbers rather than musical notes. I need to interpret the music at a particular time code to belong to a singular key being played.

Answer 1

librosa.feature.chroma_stft returns a chroma spectrogram (or chromogram). It has shape (12, n_frames). 12 for each of the 12 semitones in an octave C,C#,D..., B. Each bin in the chroma spectrogram represents the average energy of that semitone (across all octaves).

n_frames is the number of time-frames in the spectrogram. Each frame is hop_length/sr seconds long, with sr is the sample rate of the loaded audio file (possibly resampled). So to go to a given time in seconds in this spectrogram, compute frame_no = int(time / (hop_length/sr)).

Musical key

To go from chroma spectrogram to musical key (this music in A minor, or F major) can be done with supervised machine learning. A classifier would be trained on short time-windows of the chroma spectrogram (say 1-10 seconds) in order to classify the tonic (12 classes, C-B), and the mode (minor, major).

For an example, see the paper Detecting Musical Key with Supervised Learning by Robert Mahieu (2016).