How do you attach a reverb effect to AudioRecord/PCM data and save it to a file?

Question

I want to record input from the microphone, attach a reverb effect, and persist the result to a file. My use-case is an app that lets you sing a song and select different preset reverb options after recording, and then save your performance and store it on a backend server. The file that I send to the server needs to have a reverb effect applied to it.

So far I've been able to record input using AudioRecord, and I can add a reverb effect to AudioTrack to hear the reverb effect, but I'm stuck on figuring out how to save the audio with the reverb effect embedded. Here's what I have so far:

private void startRecording() {
    final int bufferSize = AudioRecord.getMinBufferSize(SAMPLE_RATE, CHANNEL_CONFIG, AUDIO_FORMAT);
    mRecorder = new AudioRecord(MediaRecorder.AudioSource.VOICE_COMMUNICATION, SAMPLE_RATE, CHANNEL_CONFIG, AUDIO_FORMAT, bufferSize);
    if(NoiseSuppressor.isAvailable()) {
        NoiseSuppressor ns = NoiseSuppressor.create(mRecorder.getAudioSessionId());
        ns.setEnabled(true);
    }

    if(AcousticEchoCanceler.isAvailable()) {
        AcousticEchoCanceler aec = AcousticEchoCanceler.create(mRecorder.getAudioSessionId());
        aec.setEnabled(true);
    }

    mPlayer = new AudioTrack(AudioManager.STREAM_MUSIC, SAMPLE_RATE, AudioFormat.CHANNEL_OUT_MONO, AUDIO_FORMAT, bufferSize, AudioTrack.MODE_STREAM);
    PresetReverb reverb = new PresetReverb(1, mPlayer.getAudioSessionId());
    reverb.setPreset(PresetReverb.PRESET_LARGEHALL);
    reverb.setEnabled(true);

    mPlayer.setAuxEffectSendLevel(1.f);

    mRecorder.startRecording();
    mPlayer.play();

    new RecordingThread(mRecorder, mPlayer, bufferSize) {

        @Override
        public void onReleased() {
            mRecorder = null;
            mPlayer = null;
        }
    }.start();
}

public class RecordingThread extends Thread {

    private static final String TAG = RecordingThread.class.getSimpleName();

    private final AudioRecord mRecorder;
    private final AudioTrack mPlayer;
    private final int mBufferSize;

    public RecordingThread(AudioRecord recorder, AudioTrack player, int bufferSize) {
        mRecorder = recorder;
        mPlayer = player;
        mBufferSize = bufferSize;
    }

    @Override
    public void run() {
        byte[] buffer = new byte[mBufferSize];

        int read;
        while ((read = mRecorder.read(buffer, 0, buffer.length)) > 0) {
            mPlayer.write(buffer, 0, buffer.length);
        }

        mRecorder.release();
        mPlayer.release();

        onReleased();

        Timber.tag(TAG);
        Timber.i("Exited with code %s", read);
    }

    public void onReleased() {
    }
}

So ideally, it'd be nice to be able to attach the PresetReverb on the AudioRecord instead of the AudioTrack, but when I do I get:

java.lang.RuntimeException: Cannot initialize effect engine for type: 47382d60-ddd8-11db-bf3a-0002a5d5c51b Error: -3

So now I'm thinking I need to pipe the PCM data from AudioRecord to some external service/library, or to modify the buffer with whatever a reverb algorithm would look like.

Answer 1

As per https://developer.android.com/reference/android/media/audiofx/PresetReverb.html

The PresetReverb is an output mix auxiliary effect and should be created on Audio session 0. In order for a MediaPlayer or AudioTrack to be fed into this effect, they must be explicitely attached to it and a send level must be specified. Use the effect ID returned by getId() method to designate this particular effect when attaching it to the MediaPlayer or AudioTrack.

Since this is designed to be used with audiotrack, this cannot be attached to AudioRecord. I think you have a similar conclusion.

I would suggest you to use this SDK https://github.com/superpoweredSDK/Low-Latency-Android-Audio-iOS-Audio-Engine. It supports android and following effects:

Effects: echo, flanger, gate, reverb, rool, whoosh, 3 band equalizer, biquad IIR filters (low-pass, high-pass, bandpass, high-shelf, low-shelf, parametric, notch)

You can also consider https://github.com/ClearVolume/ClearAudio. You can use this file to apply reverb effect https://github.com/ClearVolume/ClearAudio/blob/master/src/java/clearaudio/synthesizer/filters/ReverbFilter.java

All the best !

---

Adding some more details on how this can be done using superpoweredSDK. The superpowered SDK has a filter called SuperpoweredReverb. The way this works is the following :

Let's see this through some code:

reverb = new SuperpoweredReverb(samplerate);
reverb->enable(true);

reverb->setMix(floatValue); //  Limited to >= 0.0f and <= 1.0f.
reverb->setRoomSize(floatValue); // Limited to >= 0.0f and <= 1.0f.

SuperpoweredShortIntToFloat(intBuffer, floatBuffer, numberOfSamples);

bool res = reverb->process(floatBuffer, floatBuffer, numberOfSamples);

SuperpoweredFloatToShortInt(floatBuffer, intBuffer, numberOfSamples);

Let me try my best to explain what's happening here:

Create filter

reverb = new SuperpoweredReverb(samplerate);
reverb->enable(true);

Optional: Apply additional options to this filter such as

reverb->setMix(floatValue); //  Limited to >= 0.0f and <= 1.0f.
reverb->setRoomSize(floatValue); // Limited to >= 0.0f and <= 1.0f.

Get the source samples ; If the sample are in Integer use SuperpoweredShortIntToFloat to convert. Here intBuffer has the input audio samples and numberOfSamples is the number of samples. floatBuffer will contain float converted buffer

SuperpoweredShortIntToFloat(intBuffer, floatBuffer, numberOfSamples);

Call process to apply reverb effect. This method will apply reverb effect and copy the output to floatBuffer again

bool res = reverb->process(floatBuffer, floatBuffer, numberOfSamples);

To convert it back to Int use the following method

SuperpoweredFloatToShortInt(floatBuffer, intBuffer, numberOfSamples);

Then you can store this reverb applied audio sample anywhere you want, either in file or play it

---

Some more findings from superpoweredSDK, it also has recording functionality. See http://superpowered.com/docs/class_superpowered_recorder.html

An example:

recorder = new SuperpoweredRecorder(temporaryPath, samplerate);
recorder->start(destinationPath);

// With input samples
SuperpoweredShortIntToFloat(intBuffer, floatBuffer, numberOfSamples);

bool res = recorder->process(floatBuffer, NULL, numberOfSamples);

SuperpoweredFloatToShortInt(floatBuffer, intBuffer, numberOfSamples);