Where to start on iOS audio synth?

Question

I know this is a very broad topic, but I've been floundering around with demos and my own tests and am not sure if I'm attacking the problem correctly. So any leads on where I should start would be appreciated.

The goal is to have the app generate some synthesized sounds, per the user's settings. (This isn't the only app function, I'm not recreating Korg here, but synth is part of it.) The user would set the typical synth settings like wave, reverb, etc. then would pick when the note would play, probably with a pitch and velocity modifier.

I've played around a bit with audio unit and RemoteIO, but only barely understand what I'm doing. Before I go TOO far down that rabbit hole, I'd like to know if I'm even in the right ballpark. I know audio synth is going to be low level, but I'm hoping that maybe there are some higher level libraries out there that I can use.

If you have any pointers on where to start, and which iOS technology I should be reading about more, please let me know.

Thanks!

EDIT: let me better summarize the questions.

Are there any synth libraries already built for iOS? (commercial or Open Source - I haven't found any with numerous searches, but maybe I'm missing it.)

Are there any higher level APIs that can help generate buffers easier?

Assuming that I can already generate buffers, is there a better / easier way to submit those buffers to the iOS audio device than the RemoteIO Audio Unit?

Answer 1

This is a really good question. I sometimes ask myself the same things and I always end up using the MoMu Toolkit from the guys at Stanford. This library provides a nice callback function that connects to AudioUnits/AudioToolbox (not sure), so that all you care about is to set the sampling rate, the buffer size, and the bit depth of the audio samples, and you can easily synthesize/process anything you like inside the callback function.

I also recommend the Synthesis ToolKit (STK) for iOS that was also released by Ge Wang at Stanford. Really cool stuff to synthesize / process audio.

Every time Apple releases a new iOS version I check the new documentation in order to find a better (or simpler) way to synthesize audio, but always with no luck.

EDIT: I want to add a link to the AudioGraph source code: https://github.com/tkzic/audiograph This is a really interesting app to show the potential of AudioUnits, made by Tom Zicarelli. The code is really easy to follow, and a great way to learn about this --some would say-- convoluted process of dealing with low level audio in iOS.

Answer 2

Swift & Objective C

There's a great open source project that is well documented with videos and tutorials for both Objective-C & Swift.

AudioKit.io