iOS: Audio Unit RemoteIO AudioBuffer manipulation (i.e. sound effects from microphone)

Question

I've been playing around with Apple's aurioTouch demo which is sample code for their Audio Unit tutorial. This application allows simultaneous input/output from the mic. to speaker. It also renders a stereograph of the inputted sound from the mic.

At a really high-level of this low-level process, the sample code defines an AudioComponent (in this case RemoteIO which allows for simultaneous input/output) and there is a render callback for this Audio Unit. In the callback they do some audio filtering (a DC Rejection Filter) and visualization of the stereograph based on the AudioBuffer sound data from the mic.

My ultimate goal is to create my own custom sound distortion Audio Unit based on the input from the mic. I think the proper way to do this based on the Audio Unit tutorial is to make a second Audio Unit and connect them with an Audio Processing Graph. However, I've read that iOS doesn't allow you to register your own custom Audio Units. My questions are:

1. Can I do direct manipulation on the AudioBufferList that I have access to in the render callback from the remoteIO Audio Unit (since they already seem to be doing this and applying an audio filter on it) and create my own custom sound distortion there?
2. I've tried assigning the AudioBufferList data to a constant (a value I've seen it hold from a sample run and logging of the AudioBufferList), but it appears to do nothing.

Answer 1

The answer to your first question is yes. That is generally how it is done.

I believe that you need to manipulate the data in the pointer directly, rather than reassigning. You may want to take a look at the code in openframeworks that handles assigning buffers and passing them to a callback: https://github.com/openframeworks/openFrameworks/blob/master/addons/ofxiPhone/src/sound/ofxiPhoneSoundStream.mm

There is other code out there that you can look at, nick collins has a basic application for getting sound off the microphone and out the speaker, whist processing inbetween: http://www.cogs.susx.ac.uk/users/nc81/code.html. He also has code there that gets sample buffers out of a iPod track that may be useful to you.

Answer 2

It is true that you can not add your own custom AudioUnits to the iPhone.

The way it works it like this: The speaker drives the pull-chain of data through the system. You add a render callback to the ioUnit, as you have already done.

The callback runs whenever the speaker (bus #0) is hungry and it is your job to fill in as many samples as it has requested, in the buffer that the speaker has provided. The size of the provided buffer will be a power of two that is as close as possible to the Preferred IO Buffer Duration you specified when configuring the AudioSession.

The simplest way to do that is to take the AudioBufferList you were given and pass it to AudioUnitRender on the the microphone (bus #1). After you fill in the buffer with Render(), but before the callback returns, you can manipulate the data any way you like. For example, AurioTouch zeroes it out to mute it.

The important thing to remember is that the speaker is going to read from the actual data buffer that it passed you. It is not going to look at the AudioBufferList descriptor and check if you pointed to another databuffer. If you start changing the AudioBufferList that you were given, you will run into problems. At best, you will be ignored. At worst, you will run into memory management issues.

If you don't want to be constrained to working in just the ioData buffer, then you can use your own AudioBufferList, allocated any way you like, in any size, and ask the microphone to Render() into that. Then you can do all the manipulation you like so long as in the end you copy the results into the buffer provided by the callback (i.e. ioData->mBuffers[0].mData as it was at the time the Callback was invoked).