Which thread to use for audio decoding?

Question

When working with audio playback I am used to the following pattern:

• one disk (or network) thread which reads data from disk (or the network) and fills a ringbuffer
• one audio thread which reads data from the ringbuffer, possibly performs DSP, and writes to audio hardware (pull or push API)

This works fine, and there's no issue when working with, say, a WAV file.

Now, if the source data is encoded in a compressed format, like Vorbis or MP3, decoding takes some time.

And it seems like it's quite common to perform decoding in the disk/network thread.

But isn't this wrong design? While disk or network access blocks, some CPU time is available for decoding, but is wasted if decoding happens in the same thread.

It seems to me that if the networks becomes slow, then risks of buffer underruns are higher if decoding happens sequentially.

So, shouldn't decoding be performed in the audio thread?

In my context, I would prefer to avoid adding a dedicated decoding thread. It's for mobile platforms and SMP is pretty rare right now. But please tell if a dedicated decoding thread really makes sense to you.

Answer 1

It's more important for the audio thread to be available for playing audio smoothly than for the network thread to maintain a perfect size buffer. If you're only using two threads, then the decoding should be done on the network thread. If you were to decode on the playing thread then it's possible the time could come that you need to push more audio out to the hardware but the thread is busy decoding. It's better if you maintain a buffer of already decoded audio.

Ideally you would use three threads. One for reading the network, one for decoding, and one for playing. In our application that handles audio/video capture, recording, and streaming we have eight threads per stream (recently increased from six threads since we added new functionality recently). It's much easier for each thread to have it's own functionality and then it can appropriately measure its performance against those of it's incoming/outgoing buffers. This also benefits profiling and optimization.