How do you measure the difference between two sounds using the Web Audio API?

Question

I'm attempting to measure the difference between two sounds using an analyser node and getByteFrequencyData(). I thought that by summing the difference in each frequency bin I could come up with a single number to represent how different the two sounds were. Then I would be able to change the sounds and measure the numbers again to see if the new sound was more or less different than before.

Does getByteFrequencyData() fully encompass the representation of a sound or do I need to include other pieces of data to qualify the sound?

Here is the code I'm using:

var Spectrogram = (function(){
    function Spectrogram(ctx) {
        this.analyser = ctx.createAnalyser();
        this.analyser.fftSize = 2048;
        this.sampleRate = 512;

        this.scriptNode = ctx.createScriptProcessor(this.sampleRate, 1, 1);
        this.scriptNode.onaudioprocess = this.process.bind(this);

        this.analyser.connect(this.scriptNode);

        this.startNode = this.analyser;
        this.endNode = this.scriptNode;

        this.data = [];
    }

    Spectrogram.prototype.process = function(e) {
        var d = new Uint8Array(this.analyser.frequencyBinCount);
        this.analyser.getByteFrequencyData(d);
        this.data.push(d);

        var inputBuffer = e.inputBuffer;
        var outputBuffer = e.outputBuffer;
        for(var channel = 0; channel < outputBuffer.numberOfChannels; channel++) {
            var inputData = inputBuffer.getChannelData(channel);
            var outputData = outputBuffer.getChannelData(channel);
            for(var sample = 0; sample < inputBuffer.length; sample++) {
                outputData[sample] = inputData[sample];
            }
        }
    };

    Spectrogram.prototype.compare = function(other) {
        var fitness = 0;
        for(var i=0; i<this.data.length; i++) {
            if(other.data[i]) {
                for(var k=0; k<this.data[i].length; k++) {
                    fitness += Math.abs(this.data[i][k] - other.data[i][k]);
                }
            }
        }
        return fitness;
    }

    return Spectrogram;
})();

Answer 1

You could use the spectralFlux function provided by the Meyda package to compare two signals. Spectral flux is, according to wikipedia, "usually calculated as the 2-norm (also known as the Euclidean distance) between the two normalised spectra."

After runninng npm install --save meyda, you would do something like:

const spectralFlux = require('meyda/src/extractors/spectralFlux');

const difference = spectralFlux({
  signal: [your first signal],
  previousSignal: [your second signal]
});

Feel free to just copy the code from here so that you don't have to handle the dependency, the codebase is appropriately licensed.

It will return a coefficient of how "different" the two signals sound. You could do this in either the time domain, or the frequency domain. You'd get different numbers, but both would correlate with how "different" the sounds are from each other.

But "difference" may not describe the differences accurately enough for your use case. For instance, you may care a lot about volume differences, and not very much about timbral differences, but the spectral flux metric doesn't take that into account. You may wish to run each signal through feature extractors first, find other statistics about their properties such as their perceptual volume, their brightness, etc, and then take a weighted euclidean distance between those data, which would provide a more tailored "difference" metric to what you need for your purpose.

Happy to elaborate further, but this is already pretty long for a SO answer.