Getting averagePowerForChannel in AVPlayer

Question

how can i get the averagePowerForChannel in AVPlayer in order to make an audio visualization on my music app! ive already done the visualization part but im stuck in its engine (realtime volume channel).

i know that by using AVAudioPlayer it can be done easily using the .meteringEnabled Property but for some known reason AVPlayer is a must in my app! im actualy thinking of using AVAudioPlayer Alongside with AVPlayer to get the desired result but it sounds kind of messy workaround, how can that affect performance and stability? thanks in advance

Answer 1

I have an issue with AVPlayer visualisation for about two years. In my case it involves HLS live streaming, in that case, you won't get it running, as of my knowledge.

EDIT This will not let you access the averagePowerForChannel: method, but you will get access to the raw data and with for example FFT get the desired information.

I got it working with local playback, though. You basically wait for the players player item to have a track up and running. At that point you will need to patch an MTAudioProcessingTap into the audio mix.

The processing tap will run callbacks you specify in which you will be able to compute the raw audio data as you need.

Here is a quick example (sorry for heaving it in ObjC, though):

#import <AVFoundation/AVFoundation.h>
#import <MediaToolbox/MediaToolbox.h>

void init(MTAudioProcessingTapRef tap, void *clientInfo, void **tapStorageOut) {};
void finalize(MTAudioProcessingTapRef tap) {};
void prepare(MTAudioProcessingTapRef tap, CMItemCount maxFrames, const AudioStreamBasicDescription *processingFormat) {};
void unprepare(MTAudioProcessingTapRef tap) {};
void process(MTAudioProcessingTapRef tap, CMItemCount numberFrames, MTAudioProcessingTapFlags flags, AudioBufferList *bufferListInOut, CMItemCount *numberFramesOut, MTAudioProcessingTapFlags *flagsOut) {};

- (void)play {
    // player and item setup ...

    [[[self player] currentItem] addObserver:self forKeyPath:@"tracks" options:kNilOptions context:NULL];
}

//////////////////////////////////////////////////////

- (void)observeValueForKeyPath:(NSString *)keyPath ofObject:(id)object change:(NSDictionary *)changecontext:(void *)context
    if ([keyPath isEqualToString:@"tracks"] && [[object tracks] count] > 0) {
        for (AVPlayerItemTrack *itemTrack in [object tracks]) {
            AVAssetTrack *track = [itemTrack assetTrack];

            if ([[track mediaType] isEqualToString:AVMediaTypeAudio]) {
                [self addAudioProcessingTap:track];
                break;
            }
        }
}

- (void)addAudioProcessingTap:(AVAssetTrack *)track {
    MTAudioProcessingTapRef tap;
    MTAudioProcessingTapCallbacks callbacks;
    callbacks.version = kMTAudioProcessingTapCallbacksVersion_0;
    callbacks.clientInfo = (__bridge void *)(self);
    callbacks.init = init;
    callbacks.prepare = prepare;
    callbacks.process = process;
    callbacks.unprepare = unprepare;
    callbacks.finalize = finalise;

    OSStatus err = MTAudioProcessingTapCreate(kCFAllocatorDefault, &callbacks, kMTAudioProcessingTapCreationFlag_PostEffects, &tap);

    if (err) {
        NSLog(@"error: %@", [NSError errorWithDomain:NSOSStatusErrorDomain code:err userInfo:nil]);
        return;
    }

    AVMutableAudioMix *audioMix = [AVMutableAudioMix audioMix];

    AVMutableAudioMixInputParameters *inputParams = [AVMutableAudioMixInputParameters audioMixInputParametersWithTrack:audioTrack];
    [inputParams setAudioTapProcessor:tap];
    [audioMix setInputParameters:@[inputParams]];

    [[[self player] currentItem] setAudioMix:audioMix];
}

There is some discussion going on over on my question from over two years ago, so make sure to check it out as well.