Track camera position with RealityKit

Question

How can you track the position of the camera using RealityKit? Several examples are using SceneKit, but I found none using RealityKit. I need a function such as:

func session(_ session: ARSession, didUpdate frame: ARFrame) {
    // Do something with the new transform
    let currentTransform = frame.camera.transform
    doSomething(with: currentTransform)
}

Answer 1

Using ARView Camera Transform:

You can access the ARView Camera Transform using the following method:

var cameraTransform: Transform

The transform of the currently active camera.

So assuming your ARView was called arView you could access the Transform like so:

let cameraTransform = arView.cameraTransform

A more useful implementation however would be to enable your ARView to observe SceneEvents.Updateby making use of the following:

subscribe(to:on:_:)

func subscribe<E>(to event: E.Type, on sourceObject: EventSource? = nil, _ handler: @escaping (E) -> Void) -> Cancellable where E : Event

Which means you would have an observer of any:

event triggered once per frame interval that you can use to execute custom logic for each frame.

To do that you would: Firstly import the Combine Framework.

You would then create a Cancellable variable:

var sceneObserver: Cancellable!

Then in ViewDidLoad add something like the following:

sceneObserver = arView.scene.subscribe(to: SceneEvents.Update.self) { [unowned self] in self.updateScene(on: $0) }

Whereby each update calls the following:

/// Callback For ARView Update Events
/// - Parameter event: SceneEvents.Update
func updateScene(on event: SceneEvents.Update) {

  print(arView.cameraTransform)

}

Using ARSessionDelegate:

Alternatively you can access the ARCamera from within RealityKit by subscribing to the ARSessionDelegate e.g:

arView.session.delegate = self

And then registering for the following callback:

func session(_ session: ARSession, didUpdate frame: ARFrame)

Whereby a working example would look something like this:

extension ViewController: ARSessionDelegate {

  func session(_ session: ARSession, didUpdate frame: ARFrame) {

    guard let arCamera = session.currentFrame?.camera else { return }
    print("""
      ARCamera Transform = \(arCamera.transform)
      ARCamera ProjectionMatrix = \(arCamera.projectionMatrix)
      ARCamera EulerAngles = \(arCamera.eulerAngles)
      """)

  }

}

Hope it points you in the right direction.