Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

is it possible to get exactly the same results from tensorflow mfcc and librosa mfcc?

I'm trying to make tensorflow mfcc give me the same results as python lybrosa mfcc i have tried to match all the default parameters that are used by librosa in my tensorflow code and got a different result

this is the tensorflow code that i have used :

waveform = contrib_audio.decode_wav(
 audio_binary,
 desired_channels=1,
 desired_samples=sample_rate,
 name='decoded_sample_data')


sample_rate = 16000

transwav = tf.transpose(waveform[0])

stfts = tf.contrib.signal.stft(transwav,
  frame_length=2048,
  frame_step=512,
  fft_length=2048,
  window_fn=functools.partial(tf.contrib.signal.hann_window, 
  periodic=False), 
  pad_end=True)

spectrograms = tf.abs(stfts)
num_spectrogram_bins = stfts.shape[-1].value
lower_edge_hertz, upper_edge_hertz, num_mel_bins = 0.0,8000.0, 128
linear_to_mel_weight_matrix = 
tf.contrib.signal.linear_to_mel_weight_matrix(
num_mel_bins, num_spectrogram_bins, sample_rate, lower_edge_hertz,
   upper_edge_hertz)
mel_spectrograms = tf.tensordot(
 spectrograms, 
 linear_to_mel_weight_matrix, 1)
mel_spectrograms.set_shape(spectrograms.shape[:-1].concatenate(
linear_to_mel_weight_matrix.shape[-1:]))
log_mel_spectrograms = tf.log(mel_spectrograms + 1e-6)
mfccs = tf.contrib.signal.mfccs_from_log_mel_spectrograms(
    log_mel_spectrograms)[..., :20]

the equivalent in librosa: libr_mfcc = librosa.feature.mfcc(wav, 16000)

the following are the graphs of the results: tensorflow mfcc results

librosa mfcc results

like image 532
Eli Leszczynski Avatar asked Oct 24 '25 03:10

Eli Leszczynski


2 Answers

I'm the author of tf.signal. Sorry for not seeing this post sooner, but you can get librosa and tf.signal.stft to match if you center-pad the signal before passing it to tf.signal.stft. See this GitHub issue for more details.

like image 106
rryan Avatar answered Oct 25 '25 18:10

rryan


I spent a whole 1 day trying to make them match. Even the rryan's solution didn't work for me (center=False in librosa), but I finally found out, that TF and librosa STFT's match only for the case win_length==n_fft in librosa and frame_length==fft_length in TF. That's why rryan's colab example is working, but you can try that if you set frame_length!=fft_length, the amplitudes are very different (although visually, after plotting, the patterns look similar). Typical example - if you choose some win_length/frame_length and then you want to set n_fft/fft_length to the smallest power of 2 greater than win_length/frame_length, then the results will be different. So you need to stick with the inefficient FFT given by your window size... I don't know why it is so, but that's how it is, hopefully it will be helpful for someone.

like image 25
Stano Hrivňak Avatar answered Oct 25 '25 16:10

Stano Hrivňak



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!