cross platform sound API [closed]

Question

I'm looking into developing an application that will require live streaming of audio. I would prefer to use some cross-platform (windows/linux/BSD) open source library written in C or C++ even though writing it using the respective OSs' Sound APIs is still an option.

I have read a bit about various sound libraries, including SFML, SDL and PortAudio. Admittedly, I have not yet researched enough about Sound in FreeBSD and Linux (how similar is it between these 2?)

The main requirements will be

1. get audio from a chosen microphone/microphones to send over the network,
2. send data to a selection of output devices,
3. process the sound (filtering, cleaning up noise, multiplexing streams etc) but this can be done once I have the audio data, the library itself does NOT need to be able to do any of this.
4. have reasonably low latency

My main concern is that these mentioned APIs seem to be mainly targeted for Games (where sound is usually loaded from disk and there is not much, if any, sound recording involved rather than streamed over the network with equal importance between recording and playback.

Does anyone have any pointers/warnings/suggestions regarding these or other sound APIs or about the advantages/disadvantages of going the long way and implementing this in the respective OSs' APIs?

NOTE: while this: "Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow " DOES make the question off-topic, when one also considers "as they tend to attract opinionated answers and spam" then I do not think this question should be closed. Someone looking for such a library as described will be hard-pressed to find anything and the answers to this question practically summarize all available options. This is thus in the "generally covers...software tools commonly used by programmers;" category of accepted answers.

Answer 1

PortAudio is an excellent choice for the application you are describing -- it runs on all those platforms, is written in C, provides low-latency, and has both callback and blocking I/O options. It definitely meets your requirements, and is not particularly aimed at games. In fact, I would say there are other apis that are better for games, whereas PortAudio is better for general purpose audio I/O such as voip-like applications like yours, audio players, pro-audio applications, audio recording, software radios, etc.

Another option you might consider is RTAudio, which I am not as familiar with. My understanding is it's a bit simpler (no blocking I/O AFAIK), and has support for more platforms, including mobile OSes, though PortAudio folks are working on that.

Your question about FreeBSD vs. Linux: Linux uses ALSA, while other unixes use OSS. Both OSS and ALSA offer compatibility layers, so ALSA has OSS compatibility and vice versa, but in my experience there are bugs in both compatibility layers. Maybe things have gotten better since I last used it, though.

Some linux desktops run PulseAudio on top of ALSA. I am not sure if this is true of FreeBSD. For some reason, ALSA is configured in exclusive mode by default on most systems. While this is easy to fix in theory, the config files are weird and most users haven't done it and never will, meaning that once PulseAudio takes over, you can't access the ALSA devices directly anymore, so you may want to have a PulseAudio driver as well, unless you want to make your users change their configuration (which you might if your application requires really low latency).

I am pretty sure PortAudio supports PulseAudio, despite what it may or may not say on the website. I will ask on the mailing list and update here.

UPDATE: someone on the mailing list thought that you can use Alsa drivers to access PulseAudio. That's (great!) news to me, but there it is.

Answer 2

I consider SDL and OpenAL to be both common and widely supported. Unless these don't meet your needs I'd suggest not going to a lower level since you'll lose platform independence.

The reason they seem geared towards games is simply because that is one of the harder use cases. So if you can support games chances are you'll support anything another app will want (short of studio software).

Consider your goal of low latency. Games require a very low latency to ensure sound effects are matched well with actions on the screen. I presume this is a similar reason you want this (so your sound matches your video stream and there are no pauses in the voice channel).

BTW, your perception of games is not correct. A lot of games have user voice channels for team communication. Additionally they may include procedural sound and sound effects.

---

I found another one call SFML which includes recording support. I don't know much about it, but I've seen it can replace SDL.