How can I add core image filters to OpenTok video?

Question

When I publish a stream on iOS, TokBox uses the default camera. Is there a way to add live filters to the publisher?

I just want some easy, sample code on how to create a filter and attach it to the opentok publisher object (OTVideoCapture).

Or, if that's not the right way to do it...attaching the filter on the subscriber side works too.

How can this be done easily?

Answer 1

As I understand you want to apply filters before sending video data and also in real time. There is no easy source code here but I could tell you path.

For real time video filters you could use GPUImage framework. It has ready to use camera GPUImageVideoCamera class. So you need to create class which implements GPUImageInput (it is target in terms of GPUImage) which will produce OTVideoFrame frame from input and add it to pipeline.

Something like this:

videoCamera = [[GPUImageVideoCamera alloc] initWithSessionPreset:AVCaptureSessionPreset640x480 cameraPosition:AVCaptureDevicePositionBack];

videoCamera.outputImageOrientation = UIInterfaceOrientationPortrait;
videoCamera.horizontallyMirrorFrontFacingCamera = NO;
videoCamera.horizontallyMirrorRearFacingCamera = NO;

// filter
filter = [[GPUImageSepiaFilter alloc] init];
[videoCamera addTarget:filter];

// frame producer for OTVideoCapture
frameProducer = [[FrameProducer alloc] init];
[filter addTarget:frameProducer];

// camera view to show what we record
[filter addTarget:filterView];

Also you need custom implementation of OTVideoCapture protocol for OpenTok itself. You could use TBExampleVideoCapture from Lets-Build-OTPublisher sample as a start point. You need to replace camera code with above GPUImageVideoCamera camera code to use filters in real time.