How to convert CMSampleBuffer to Data in Swift?

Question

I need to convert CMSampleBuffer to Data format. I am using one Third party framework for audio related task. That framework gives me the streaming (i.e Real Time audio) audio in CMSampleBuffer object.

Like this:

func didAudioStreaming(audioSample: CMSampleBuffer!) {
    //Here I need to conver this to Data format. 
    //Because I am using GRPC framework for Audio Recognization, 
}

Please provide me the steps to convert the CMSampleBuffer to Data.

FYI

    let formatDesc:CMFormatDescription? = CMSampleBufferGetFormatDescription(audioSample)

    <CMAudioFormatDescription 0x17010d890 [0x1b453ebb8]> {
    mediaType:'soun' 
    mediaSubType:'lpcm' 
    mediaSpecific: {
        ASBD: {
            mSampleRate: 16000.000000 
            mFormatID: 'lpcm' 
            mFormatFlags: 0xc 
            mBytesPerPacket: 2 
            mFramesPerPacket: 1 
            mBytesPerFrame: 2 
            mChannelsPerFrame: 1 
            mBitsPerChannel: 16     } 
        cookie: {(null)} 
        ACL: {(null)}
        FormatList Array: {(null)} 
    } 
    extensions: {(null)}
}

Answer 1

Try below code to convert CMSampleBuffer to NSData.

let imageBuffer = CMSampleBufferGetImageBuffer(sampleBuffer)
CVPixelBufferLockBaseAddress(imageBuffer!, CVPixelBufferLockFlags(rawValue: 0))
let bytesPerRow = CVPixelBufferGetBytesPerRow(imageBuffer!)
let height = CVPixelBufferGetHeight(imageBuffer!)
let src_buff = CVPixelBufferGetBaseAddress(imageBuffer!)
let data = NSData(bytes: src_buff, length: bytesPerRow * height)
CVPixelBufferUnlockBaseAddress(imageBuffer!, CVPixelBufferLockFlags(rawValue: 0))

EDIT-

For AudioBuffer use below code -

var audioBufferList = AudioBufferList()
var data = Data()
var blockBuffer : CMBlockBuffer?

CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer(sampleBuffer, nil, &audioBufferList, MemoryLayout<AudioBufferList>.size, nil, nil, 0, &blockBuffer)

let buffers = UnsafeBufferPointer<AudioBuffer>(start: &audioBufferList.mBuffers, count: Int(audioBufferList.mNumberBuffers))

for audioBuffer in buffers {
    let frame = audioBuffer.mData?.assumingMemoryBound(to: UInt8.self)
    data.append(frame!, count: Int(audioBuffer.mDataByteSize))
}

Answer 2

Using CMSampleBufferGetAudioBufferListWithRetainedBlockBuffer will require to call at some point CFRelease(blockBuffer) because the buffer is retained and if not released the pool of buffers will become eventually empty and no new CMSampleBuffer will be generated.

I'd suggest to get directly the data using the following:

CMBlockBufferRef blockBuffer = CMSampleBufferGetDataBuffer(sampleBuffer);
size_t lengthAtOffset;
size_t totalLength;
char *data;
CMBlockBufferGetDataPointer(blockBuffer, 0, &lengthAtOffset, &totalLength, &data);

NSData *audioData = [NSData dataWithBytes:data length:totalLength];