Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Playing audio while applying video filter in GPUImage

I've googled this question but did not find any solution.

So, my problem is, I'm applying video filter using GPUImage to a video file. At same time, sound of that video is not playing. I know that sound playing is not supported in GPUImage while applying filters.

So, how can I achieve that?

like image 463
Bharat Nakum Avatar asked Mar 16 '23 02:03

Bharat Nakum


1 Answers

You can achieve sound playing by adding support for AVPlayer like this.

  1. In GPUImageMovie.h,

    @property(readwrite, nonatomic) BOOL playSound;

  2. In GPUImageMovie.m, update @interface GPUImageMovie () like this.

    @interface GPUImageMovie() <AVPlayerItemOutputPullDelegate> 
    {
        // Add this
        BOOL hasAudioTrack;
    
        .........
        ........
    
       // Add all below three lines
       AVPlayer *theAudioPlayer;
       CFAbsoluteTime startActualFrameTime;
       CGFloat currentVideoTime;
    }
    
  3. After that, in -(void)startProcessing method, add following lines.

    - (void)startProcessing
    {
       // Add this
       currentVideoTime = 0.0f;
    
       .....
       .....
    
       // Add this
       if (self.playSound)
       {
           [self setupSound];
       }
    
       GPUImageMovie __block *blockSelf = self;
    
       [inputAsset loadValuesAsynchronouslyForKeys:[NSArray arrayWithObject:@"tracks"] completionHandler: ^{
        runSynchronouslyOnVideoProcessingQueue(^{
    
                .........
                .........
    
                // Add this
                startActualFrameTime = CFAbsoluteTimeGetCurrent() - currentVideoTime;
    
                .......
                .......
            });
       }];
    }
    
  4. Again, In -(AVAssetReader*)createAssetReader update it,

    - (AVAssetReader*)createAssetReader
    {
        .......
        .......  
        BOOL shouldRecordAudioTrack = (([audioTracks count] > 0) &&  (self.audioEncodingTarget != nil));
    
        // Add this
        hasAudioTrack = ([audioTracks count] > 0);
    
        .........
    }
    
  5. In - (void)processAsset,

    - (void)processAsset
    {
    
       .......
       .......
    
       if ([reader startReading] == NO)
       {
          NSLog(@"Error reading from file at URL: %@", self.url);
          return;
       }
    
       // Add this
       if (self.playSound && hasAudioTrack)
       {
          [theAudioPlayer seekToTime:kCMTimeZero];
          [theAudioPlayer play];
       }
    
       .........
       .........
    }
    
  6. In -(void)endProcessing,

    - (void)endProcessing
    {
       .........
       .........
    
       if (self.playerItem && (displayLink != nil))
       {
          [displayLink invalidate]; // remove from all run loops
          displayLink = nil;
       } 
    
       // Add this
       if (theAudioPlayer != nil)
       {
          [theAudioPlayer pause];
          [theAudioPlayer seekToTime:kCMTimeZero];
       }
    
       .........
       .........
    }
    
  7. Add new method -(void)setupSound at last,

    // Add this
    - (void)setupSound
    {
       if (theAudioPlayer != nil)
       {
           [theAudioPlayer pause];
           [theAudioPlayer seekToTime:kCMTimeZero];
           theAudioPlayer = nil;
       }
    
       theAudioPlayer = [[AVPlayer alloc] initWithURL:self.url];
    }
    
  8. At last, when you create your GPUImageMovie object, don't forget to set playSound flag to YES.

    Like this : movieFile.playSound = YES;

    Hope this helps.

like image 151
Bharat Nakum Avatar answered Mar 25 '23 05:03

Bharat Nakum