Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ARCore – Session, Frame, Camera and Pose

I'm studying the ARCore References, Develop, make the course from Coursera, and read, understood and learn from the Samples.

But I still missing some definition with some real use examples.

What is a session? Every time that I need a ARCore use I need a session? Session always has a camera connect so I can see and draw/renderer my 3D models in the screen? Can I do this without a Session?

Camera has a getPose and Frame has a GetPose, what are the diferences between they?

I thought about make this questions split but somehow I know that they are all connected. Sessions, CameraAr, Frame and Pose.

like image 963
Canato Avatar asked Aug 09 '18 09:08

Canato


People also ask

What is pose in ARCore?

public class Pose. Represents an immutable rigid transformation from one coordinate space to another. As provided from all ARCore APIs, Poses always describe the transformation from object's local coordinate space to the world coordinate space (see below).

What is ARCore camera?

ARCore is Google's platform for building augmented reality experiences. Using different APIs, ARCore enables your phone to sense its environment, understand the world and interact with information. Some of the APIs are available across Android and iOS to enable shared AR experiences.

Is ARKit or ARCore better?

ARKit tends to perform better than ARCore in terms of image tracking and recognition. If you intend to create AR apps that track user gestures to manipulate on-screen images, ARKit will usually be the more efficient option. It translates movements into data faster than Google's alternative.

Is ARCore deprecated?

Caution: The ARCore SDK for Unity is deprecated, and no longer supported in Unity 2020 and later. This SDK should only be used by developers working on existing projects which are unable to migrate to Unitys AR Foundation.


1 Answers

About ArSession

ArSession is the most crucial element in AR puzzle. Session manages AR system state and handles the session lifecycle. Session class is the main entry point to the ARCore API. This class allows the user to create a session, configure it, start or stop it and, most importantly, receive ArFrames that allow access to ARCamera image and device Pose.

In order to use ARCore you need an ArSession. ARCore doesn't render 3D models (Renderables). This job is for Sceneform framework.

enter image description here

Code's example:

private Session mSession;
Config config = new Config(mSession);

if (!mSession.isSupported(config)) {
    showSnackbarMessage("This phone doesn't support AR", true);
}
mSession.configure(config);

Also Session's configuration can include nested classes:

  • Config.AugmentedFaceMode (Selects the behaviour of Augmented Faces subsystem)
  • Config.CloudAnchorMode (The cloud anchor mode in Config)
  • Config.FocusMode (Selects the desired behaviour of the camera focus subsystem)
  • Config.LightEstimationMode (Select the behaviour of the lighting estimation subsystem)
  • Config.PlaneFindingMode (Select the behaviour of the plane detection subsystem)
  • Config.UpdateMode (Selects the behaviour of update())

About Pose

Pose represents an immutable rigid transformation from one coordinate space to another. As provided from all ARCore APIs, Poses always describe the transformation from object's local coordinate space to the world coordinate space. The transformation is defined using a quaternion rotation about the origin followed by a translation.

Code's example:

float[] position = { 0, 0, -2.2 };      //  { x, y, z } position 
float[] rotation = { 0, 0, 0, 1 };      //  { x, y, z, w } quaternion rotation

Session session = arFragment.getArSceneView().getSession();
Anchor myAnchor = session.createAnchor(new Pose(position, rotation));

About ArCamera

ArCamera represents a virtual camera, which determines the perspective through which the scene is viewed. If the camera is part of an ArSceneView, then the camera automatically tracks the Camera Pose from ARCore. ArCamera is a long-lived object and the properties of camera are updated every time Session.update() is called. Camera class provides information about the camera that is used to capture images and additional info inside each ArFrame.

Code's example:

// Shared camera access with ARCore

sharedSession = new Session(this, EnumSet.of(Session.Feature.SHARED_CAMERA))
sharedCamera = sharedSession.getSharedCamera();
cameraId = sharedSession.getCameraConfig().getCameraId();

About ArFrame

When ARCore's understanding of the environment changes, it adjusts its model of the world to keep things consistent. When this happens, the numerical location (coordinates) of the ARCamera and ARAnchors can change significantly to maintain appropriate relative positions of the physical locations they represent.These changes mean that every ArFrame should be considered to be in a completely unique world coordinate space. The numerical coordinates of ARAnchors and the ARCamera should never be used outside the rendering frame during which they were retrieved.

Every ArFrame stores the following info about ARCore's state::

  • RGB image itself
  • Tracking status
  • The pose of the camera relative to the world
  • Estimated lighting parameters
  • Information on updates to objects (like Point Clouds)

Code's example:

private void onUpdateFrame(FrameTime frameTime) {

    Frame frame = arFragment.getArSceneView().getArFrame();

    // .............
}


Additionally, you can read this useful post.

like image 128
Andy Jazz Avatar answered Sep 19 '22 03:09

Andy Jazz