Adding custom view to ARKit

Question

I just started looking at ARKitExample from apple and I am still studying. I need to do like interactive guide. For example, we can detect something (like QRCode), in that area, can I show with 1 label ?

Is it possible to add custom view (like may be UIVIew, UIlabel) to surface?

Edit

I saw some example to add line. I will need to find how to add additional view or image.

    let mat = SCNMatrix4FromMat4(currentFrame.camera.transform)
    let dir = SCNVector3(-1 * mat.m31, -1 * mat.m32, -1 * mat.m33)
    let currentPosition = pointOfView.position + (dir * 0.1)

    if button!.isHighlighted {
        if let previousPoint = previousPoint {
            let line = lineFrom(vector: previousPoint, toVector: currentPosition)
            let lineNode = SCNNode(geometry: line)
            lineNode.geometry?.firstMaterial?.diffuse.contents = lineColor
            sceneView.scene.rootNode.addChildNode(lineNode)
        }
    }

I think this code should be able to add custom image. But I need to find the whole sample.

func updateRenderer（_ frame： ARFrame）{
    drawCameraImage（withPixelBuffer：frame.capturedImage)
    let viewMatrix = simd_inverse(frame.came.transform)
    let prijectionMatrix = frame.camera.prijectionMatrix
    updateCamera(viewMatrix, projectionMatrix) 
    updateLighting(frame.lightEstimate?.ambientIntensity)
    drawGeometry(forAnchors: frame.anchors)
}

Answer 1

ARKit isn't a rendering engine — it doesn't display any content for you. ARKit provides information about real-world spaces for use by rendering engines such as SceneKit, Unity, and any custom engine you build (with Metal, etc), so that they can display content that appears to inhabit real-world space. Thus, any "how do I show" question for ARKit is actually a question for whichever rendering engine you use with ARKit.

SceneKit is the easy out-of-the-box, no-additional-software-required way to display 3D content with ARKit, so I presume you're asking about that.

SceneKit can't render a UIView as part of a 3D scene. But it can render planes, cubes, or other shapes, and texture-map 2D content onto them. If you want to draw a text label on a plane detected by ARKit, that's the direction to investigate — follow the example's, um, example to create SCNPlane objects corresponding to detected ARPlaneAnchors, get yourself an image of some text, and set that image as the plane geometry's diffuse contents.