Technique World tracking performance is being affected by resource constraints

Question

While running an ARKit session with world tracking enabled, the Xcode console shows log messages about (I presume: reduced) tracking performance, even though

• The AR Session is in Normal tracking state,

• I am using the device in a well-lit office room with plenty of "features" to detect, and

• The device moves only subtly.

TLDR: I want to understand what could be causing them, what impact they could have, and how to prevent them, or (re)act to them when they do occur—NB. not simply hide these errors.

[Technique] World tracking performance is being affected by resource constraints [0]
[Technique] World tracking performance is being affected by resource constraints [1]

The Console app shows that these are coming from the library ARKit and fall in the logging category Technique. Although they sound like warnings, the Console app shows their type as errors.

As expected while using world tracking, the Console app does show a lot of CoreMotion logs around the time of the errors, but those lines do not seem to contain any errors, warnings, or other info that helps me diagnose what's going on.

The moment the errors show up in the log, there are no delegate callbacks, but eventually (anything between 5 seconds or 50 seconds) the screen will freeze with the callback session failed:

Error Domain=com.apple.arkit.error Code=200 "World tracking failed." UserInfo={NSLocalizedDescription=World tracking failed., NSLocalizedFailureReason=World tracking cannot determine the device's position.}

The ARKit source/documentation doesn't provide any hints on what "resource constraints" might have caused the inability to determine the device's position, it just reads:

World tracking has encountered a fatal error.

I've tried (without success) to stop the warnings from appearing by:

• Resetting the session tracking: still errors,

• Resetting the session with removal of all ARAnchors: still errors,

• Disabling plane detection (once no longer needed): still errors.

Pausing the AR Session silences the warnings (makes sense, since it means the device stops tracking its movement while paused), but when resuming the session, the warnings return.

When closing the session and re-creating it (i.e. dismiss VC and recreate), while not having moved the camera (or changed lighting) the problem doesn't always re-occur.

My best guess is that blinking TL-lights are the cause for the tracking performance warnings, given Apple's explanation how world tracking works:

… visual-inertial odometry. This process combines information from the iOS device’s motion sensing hardware with computer vision analysis of the scene visible to the device’s camera. ARKit recognizes notable features in the scene image, tracks differences in the positions of those features across video frames, and compares that information with motion sensing data.

(iPhone 6S, iOS 11 beta 4, no other apps running in the background)

---

Questions:

• What is the difference between [0] and [1]?

• What can cause these errors?

• What impact do they have while the ARSession hasn't failed (yet)? I'm assuming we'll see "jumpy" models as the world coordinates supplied by the frame updates aren't accurate—NB. after resetting/stopping ARAnchor tracking the errors still occurred.

• Will we know if the AR session is about to fail, or will it be unexpected? (Again, the ARSession doesn't fail immediately after the tracking state changes to "limited")

• Is there some way to deal with this, e.g. freeing up some of these mentioned "resources"—or other ways to avoid running into these constraints?

Answer 1

Only allowing Portrait Device Orientation resolved this error for me.

Answer 2

I think this is xcode 9.1 bug. If you are use SceneKit: find where you set .backgroundColor = "your color" for ARSCNView or subclass. Comment this row and clean project.

Answer 3

This error happened to me because I manually added a view controller on the storyboard and added an ARSKView to it. This adds it as a child to a UIView which is by default a child view in a UIViewController. Therefore to support landscape and portrait orientation I had to add Auto-layout constraints, which for some reason triggered this error.

When I looked at the stock AR project, the ARSKView is an immediate child of the view controller, and it supports all view sizes and orientations out of the box without auto-layout constraints. When I set it up this way, the errors disappeared.