How to detect a movie being played in a WKWebView?

Question

I'd like to know wether it's possible to detect a movie being played in the WKWebView?

Additionally I'd like to know the exact URL of the opened stream?

Answer 1

Since the solution(s) to this question required a lot of research and different approaches, I'd like to document it here for others to follow my thoughts. If you're just interested in the final solution, look for some fancy headings.

The app I started with, was pretty simple. It's a Single-View Application that imports WebKit and opens a WKWebView with some NSURL:

import UIKit
import WebKit
class ViewController: UIViewController {
    var webView: WKWebView!

    override func viewDidAppear(animated: Bool) {
        webView = WKWebView()
        view = webView
        let request = NSURLRequest(URL: NSURL(string: "http://tinas-burger.tumblr.com/post/133991473113")!)
        webView.loadRequest(request) 
    }
}

The URL includes a video that is (kind of) protected by JavaScript. I really haven't seen the video yet, it was just the first I discovered. Remember to add NSAppTransportSecurity and NSAllowsArbitraryLoads to your Info.plist or you will see a blank page.

WKNavigationDelegate

The WKNavigationDelegate won't notify you about a video being played. So setting webView.navigationDelegate = self and implementing the protocol won't bring you the desired results.

NSNotificationCenter

I assumed that there must be an event like SomeVideoPlayerDidOpen. Unfortunately there wasn't any, but it might have a SomeViewDidOpen event, so I started inspecting the view hierarchy:

UIWindow
    UIWindow
        WKWebView
            WKScrollView
            ...
        ...
UIWindow
    UIWindow
        UIView
            AVPlayerView
        UITransitionView
            UIView
                UIView
                    UIView
                        ...
                    UIView
                        ...
                    AVTouchIgnoringView
                        ...

As expected there will be an additional UIWindow added which might have an event and hell yes it does have!

I extended viewDidAppear: by adding a new observer:

NSNotificationCenter.defaultCenter().addObserver(self, selector: "windowDidBecomeVisible:", name: UIWindowDidBecomeVisibleNotification, object: nil)

And added the corresponding method:

func windowDidBecomeVisible(notification: NSNotification) {
    for mainWindow in UIApplication.sharedApplication().windows {
        for mainWindowSubview in mainWindow.subviews {
            // this will print:
            // 1: `WKWebView` + `[WKScrollView]`
            // 2: `UIView` + `[]`
            print("\(mainWindowSubview) \(mainWindowSubview.subviews)")
}

As expected it returns the view hierarchy as we inspected earlier. But unfortunately it seems like the AVPlayerView will be created later.

If you trust your application that the only UIWindow it'll open is the media player, you're finished at this point. But this solution wouldn't let me sleep at night, so let's go deeper...

Injecting An Event

We need to get notified about the AVPlayerView being added to this nameless UIView. It seems pretty obvious that AVPlayerView must be a subclass of UIView but since it's not officially documented by Apple I checked the iOS Runtime Headers for AVPlayerView and it definitely is a UIView.

Now that we know that AVPlayerView is a subclass of UIView it will probably added to the nameless UIView by calling addSubview:. So we'd have to get notified about a view that was added. Unfortunately UIView doesn't provide an event for this to be observed. But it does call a method called didAddSubview: which could be very handy.

So let's check wether a AVPlayerView will be added somewhere in our application and send a notification:

let originalDidAddSubviewMethod = class_getInstanceMethod(UIView.self, "didAddSubview:")
let originalDidAddSubviewImplementation = method_getImplementation(originalDidAddSubviewMethod)

typealias DidAddSubviewCFunction = @convention(c) (AnyObject, Selector, UIView) -> Void
let castedOriginalDidAddSubviewImplementation = unsafeBitCast(originalDidAddSubviewImplementation, DidAddSubviewCFunction.self)

let newDidAddSubviewImplementationBlock: @convention(block) (AnyObject!, UIView) -> Void = { (view: AnyObject!, subview: UIView) -> Void in
    castedOriginalDidAddSubviewImplementation(view, "didAddsubview:", subview)

    if object_getClass(view).description() == "AVPlayerView" {
        NSNotificationCenter.defaultCenter().postNotificationName("PlayerWillOpen", object: nil)
    }
}

let newDidAddSubviewImplementation = imp_implementationWithBlock(unsafeBitCast(newDidAddSubviewImplementationBlock, AnyObject.self))
method_setImplementation(originalDidAddSubviewMethod, newDidAddSubviewImplementation)

Now we can observe the notification and receive the corresponding event:

NSNotificationCenter.defaultCenter().addObserver(self, selector: "playerWillOpen:", name: "PlayerWillOpen", object: nil)

func playerWillOpen(notification: NSNotification) {
    print("A Player will be opened now")
}

Better notification injection

Since the AVPlayerView won't get removed but only deallocated we'll have to rewrite our code a little bit and inject some notifications to the AVPlayerViewController. That way we'll have as many notifications as we want, e.g.: PlayerWillAppear and PlayerWillDisappear:

let originalViewWillAppearMethod = class_getInstanceMethod(UIViewController.self, "viewWillAppear:")
let originalViewWillAppearImplementation = method_getImplementation(originalViewWillAppearMethod)

typealias ViewWillAppearCFunction = @convention(c) (UIViewController, Selector, Bool) -> Void
let castedOriginalViewWillAppearImplementation = unsafeBitCast(originalViewWillAppearImplementation, ViewWillAppearCFunction.self)

let newViewWillAppearImplementationBlock: @convention(block) (UIViewController!, Bool) -> Void = { (viewController: UIViewController!, animated: Bool) -> Void in
    castedOriginalViewWillAppearImplementation(viewController, "viewWillAppear:", animated)

    if viewController is AVPlayerViewController {
        NSNotificationCenter.defaultCenter().postNotificationName("PlayerWillAppear", object: nil)
    }
}

let newViewWillAppearImplementation = imp_implementationWithBlock(unsafeBitCast(newViewWillAppearImplementationBlock, AnyObject.self))
method_setImplementation(originalViewWillAppearMethod, newViewWillAppearImplementation)

let originalViewWillDisappearMethod = class_getInstanceMethod(UIViewController.self, "viewWillDisappear:")
let originalViewWillDisappearImplementation = method_getImplementation(originalViewWillDisappearMethod)

typealias ViewWillDisappearCFunction = @convention(c) (UIViewController, Selector, Bool) -> Void
let castedOriginalViewWillDisappearImplementation = unsafeBitCast(originalViewWillDisappearImplementation, ViewWillDisappearCFunction.self)

let newViewWillDisappearImplementationBlock: @convention(block) (UIViewController!, Bool) -> Void = { (viewController: UIViewController!, animated: Bool) -> Void in
    castedOriginalViewWillDisappearImplementation(viewController, "viewWillDisappear:", animated)

    if viewController is AVPlayerViewController {
        NSNotificationCenter.defaultCenter().postNotificationName("PlayerWillDisappear", object: nil)
    }
}

let newViewWillDisappearImplementation = imp_implementationWithBlock(unsafeBitCast(newViewWillDisappearImplementationBlock, AnyObject.self))
method_setImplementation(originalViewWillDisappearMethod, newViewWillDisappearImplementation)

Now we can observe these two notifications and are good to go:

 NSNotificationCenter.defaultCenter().addObserver(self, selector: "playerWillAppear:", name: "PlayerWillAppear", object: nil)
NSNotificationCenter.defaultCenter().addObserver(self, selector: "playerWillDisappear:", name: "PlayerWillDisappear", object: nil)

func playerWillAppear(notification: NSNotification) {
    print("A Player will be opened now")
}

func playerWillDisappear(notification: NSNotification) {
    print("A Player will be closed now")
}

URL of the video

I spent a couple of hours digging some iOS Runtime Headers to guess where I could find the URL pointing to the video, but I couldn't manage to find it. When I was digging into some source files of WebKit itself, I had to give up and accept that there's no easy way to do it, although I believe it's somewhere hidden and can be reached, but most likely only with a lot of effort.

Answer 2

I tried to add (inject) some user script to the WKWebView, this way is a little bit safer than method swizzling:

Here is related code:

let contentController = WKUserContentController()

if let jsSource = NSBundle.mainBundle().URLForResource("video_play_messenger", withExtension: "js"),
    let jsSourceString = try? String(contentsOfURL: jsSource) {
    let userScript = WKUserScript(source: jsSourceString, injectionTime: .AtDocumentEnd, forMainFrameOnly: true)

    contentController.addUserScript(userScript)
    contentController.addScriptMessageHandler(self, name: "callbackHandler")
}

let webConfiguration = WKWebViewConfiguration()
webConfiguration.userContentController = contentController

webView = WKWebView(frame: CGRect.zero, configuration: webConfiguration)
let request = NSURLRequest(URL: NSURL(string: "URL_FOR_VIDEO")!)
webView.loadRequest(request)

For the controller of WKWebView, conform to WKScriptMessageHandler and implement this method:

func userContentController(userContentController: WKUserContentController, didReceiveScriptMessage message: WKScriptMessage) {
    if message.name == "callbackHandler" {
        if let messageString = message.body as? String where messageString == "VideoIsPlaying" {
            // Vide is being played
        }
    }
}

Add video_play_messenger.js to your project:

function videoTags() {
    return document.getElementsByTagName("video");
}

function setupVideoPlayingHandler() {
    try {
        var videos = videoTags()
        for (var i = 0; i < videos.length; i++) {
            videos.item(i).onplaying = function() {
                webkit.messageHandlers.callbackHandler.postMessage("VideoIsPlaying");
            }
        }
    } catch (error) {
        console.log(error);
    }
}

function setupVidePlayingListener() {
    // If we have video tags, setup onplaying handler
    if (videoTags().length > 0) {
        setupVideoPlayingHandler();
        return
    }

    // Otherwise, wait for 100ms and check again.
    setTimeout(setupVidePlayingListener, 100);
}

setupVidePlayingListener();

Reference: http://www.kinderas.com/technology/2014/6/15/wkwebview-and-javascript-in-ios-8-using-swift