How to play 360 video on the iOS device

Question

Looking through different web sites and analyzing different resources I found out that for playing 360 videos on the iPhone you should use 3-d party lib (Panorama). But I'm really interested in how it is possible to do it by your own. Because standard iOS elements does not support such functionality.

Please give some advices about approaches that should i use to create own player for 360 videos.

Answer 1

You can do it using scenekit, without any external libraries or dependencies.

Create a SCNScene and map your camera to the device movements. The cameras are a bit offsetted as so to map one per eye and create a 3D stereoscopic effect.

override func viewDidLoad() {
    super.viewDidLoad()
    leftSceneView?.backgroundColor = UIColor.blackColor()
    rightSceneView?.backgroundColor = UIColor.whiteColor()

    // Create Scene
    scene = SCNScene()
    leftSceneView?.scene = scene
    rightSceneView?.scene = scene

    // Create cameras
    let camX = 0.0 as Float
    let camY = 0.0 as Float
    let camZ = 0.0 as Float
    let zFar = 50.0

    let leftCamera = SCNCamera()
    let rightCamera = SCNCamera()

    leftCamera.zFar = zFar
    rightCamera.zFar = zFar

    let leftCameraNode = SCNNode()
    leftCameraNode.camera = leftCamera
    leftCameraNode.position = SCNVector3(x: camX - 0.5, y: camY, z: camZ)

    let rightCameraNode = SCNNode()
    rightCameraNode.camera = rightCamera
    rightCameraNode.position = SCNVector3(x: camX + 0.5, y: camY, z: camZ)

    camerasNode = SCNNode()
    camerasNode!.position = SCNVector3(x: camX, y:camY, z:camZ)
    camerasNode!.addChildNode(leftCameraNode)
    camerasNode!.addChildNode(rightCameraNode)

    camerasNode!.eulerAngles = SCNVector3Make(degreesToRadians(-90.0), 0, 0)

    cameraRollNode = SCNNode()
    cameraRollNode!.addChildNode(camerasNode!)

    cameraPitchNode = SCNNode()
    cameraPitchNode!.addChildNode(cameraRollNode!)

    cameraYawNode = SCNNode()
    cameraYawNode!.addChildNode(cameraPitchNode!)

    scene!.rootNode.addChildNode(cameraYawNode!)

    leftSceneView?.pointOfView = leftCameraNode
    rightSceneView?.pointOfView = rightCameraNode

    // Respond to user head movement. Refreshes the position of the camera 60 times per second.
    motionManager = CMMotionManager()
    motionManager?.deviceMotionUpdateInterval = 1.0 / 60.0
    motionManager?.startDeviceMotionUpdatesUsingReferenceFrame(CMAttitudeReferenceFrame.XArbitraryZVertical)

    leftSceneView?.delegate = self

    leftSceneView?.playing = true
    rightSceneView?.playing = true        
}

Update the camera position in the sceneRenderer:

func renderer(aRenderer: SCNSceneRenderer, updateAtTime time: NSTimeInterval){

    // Render the scene
    dispatch_async(dispatch_get_main_queue()) { () -> Void in
        if let mm = self.motionManager, let motion = mm.deviceMotion {
            let currentAttitude = motion.attitude

            var orientationMultiplier = 1.0
            if(UIApplication.sharedApplication().statusBarOrientation == UIInterfaceOrientation.LandscapeRight){ orientationMultiplier = -1.0}

            self.cameraRollNode!.eulerAngles.x = Float(currentAttitude.roll * orientationMultiplier)
            self.cameraPitchNode!.eulerAngles.z = Float(currentAttitude.pitch)
            self.cameraYawNode!.eulerAngles.y = Float(currentAttitude.yaw)

        }
    }
}

Here is some code to add a SCNSphere displaying an AVPlayer.

func play(){

    //let fileURL: NSURL? = NSURL(string: "http://www.kolor.com/360-videos-files/noa-neal-graffiti-360-music-video-full-hd.mp4")
    let fileURL: NSURL? = NSURL.fileURLWithPath(NSBundle.mainBundle().pathForResource("vr", ofType: "mp4")!)

    if (fileURL != nil){
        videoSpriteKitNode =  SKVideoNode(AVPlayer: AVPlayer(URL: fileURL!))
        videoNode = SCNNode()
        videoNode!.geometry = SCNSphere(radius: 30)

        let spriteKitScene = SKScene(size: CGSize(width: 2500, height: 2500))
        spriteKitScene.scaleMode = .AspectFit

        videoSpriteKitNode!.position = CGPoint(x: spriteKitScene.size.width / 2.0, y: spriteKitScene.size.height / 2.0)
        videoSpriteKitNode!.size = spriteKitScene.size

        spriteKitScene.addChild(videoSpriteKitNode!)

        videoNode!.geometry?.firstMaterial?.diffuse.contents = spriteKitScene
        videoNode!.geometry?.firstMaterial?.doubleSided = true

        // Flip video upside down, so that it's shown in the right position
        var transform = SCNMatrix4MakeRotation(Float(M_PI), 0.0, 0.0, 1.0)
        transform = SCNMatrix4Translate(transform, 1.0, 1.0, 0.0)

        videoNode!.pivot = SCNMatrix4MakeRotation(Float(M_PI_2), 0.0, -1.0, 0.0)
        videoNode!.geometry?.firstMaterial?.diffuse.contentsTransform = transform
        videoNode!.position = SCNVector3(x: 0, y: 0, z: 0)

        scene!.rootNode.addChildNode(videoNode!)
        videoSpriteKitNode!.play()

        playingVideo = true   
    }
}

I've put together a project on github to show how, with instructions that should be clear !

Works in VR too with a google cardboard.

https://github.com/Aralekk/simple360player_iOS

Answer 2

Apart from Aralekk's repo, I've found hanton's repo useful when creating my own video 360° player. Here is the link to my repo and here is related blogpost.