Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

ARKit – sceneView renders its content at 120 fps (but I need 30 fps)

I'm developing ARKit app along with Vision/AVKit frameworks. My app recognizes hand gestures ("Victory", "Okey", "Fist" gestures) for controlling a video. So I'm using MLModel for classification of my hand gestures.

App works fine but view's content is rendered at 120 fps. I do not need such a frame rate. It's too much for my app and it's a heavy burden for CPU. I tried to lower a frame rate to 30 fps using SceneKit's instance property:

var preferredFramesPerSecond: Int { get set }

but my frame rate is still the same – 120 fps.

enter image description here

Here's how I made it:

import UIKit
import AVKit
import SceneKit
import ARKit
import Vision

class ViewController: UIViewController, ARSCNViewDelegate {

    @IBOutlet var sceneView: ARSCNView! 
    var avPlayerView = AVPlayerViewController()
    private let player = AVQueuePlayer()
    let clips = ["AA", "BB", "CC"]
    private var token: NSKeyValueObservation?
    var number: Int = 0
    var once: Bool = true    
    let dispatchQueueML = DispatchQueue(label: "net.aaa.iii") 
    var visionRequests = [VNRequest]()

    override func viewDidLoad() {
        super.viewDidLoad()
        sceneView.delegate = self
        sceneView.showsStatistics = true
        sceneView.preferredFramesPerSecond = 30     // HERE IT GOES
        sceneView.rendersContinuously = true
    
        let scene = SCNScene()
        sceneView.scene = scene
        sceneView.scene.background.contents = UIColor.black.withAlphaComponent(0)
    
        guard let selectedModel = try? VNCoreMLModel(for: handsModel().model) else {
            fatalError("Couldn't load a model.")
        }
    
        let classificationRequest = VNCoreMLRequest(model: selectedModel, 
                                        completionHandler: classificationCompleteHandler)
        classificationRequest.imageCropAndScaleOption = VNImageCropAndScaleOption.centerCrop
        visionRequests = [classificationRequest]       
    }

    override func viewDidAppear(_ animated: Bool) {
        super.viewDidAppear(animated)
        self.addAllVideosToPlayer()
        present(avPlayerView, animated: true,
                            completion: { self.player.play() })
    
        let configuration = ARWorldTrackingConfiguration()
        configuration.isAutoFocusEnabled = false
        sceneView.session.run(configuration)
    }

    func addAllVideosToPlayer() {
        avPlayerView.player = player
        // .........................
    }
    func toggleNextVideo() {
        // ..............
    }
    func togglePreviousVideo() {
        // ..............
    }

    // ..............................
    // ..............................

    DispatchQueue.main.async {
        if (topPredictionScore != nil && topPredictionScore! > 0.01) {  
            if (topPredictionName == "FistGesture") && (self.once == false) {
                self.once = true
            }
            if (topPredictionName == "OkeyGesture") && (self.once == true) {
                self.toggleNextVideo()
                self.once = false
            }
        }
    }
}

Here's what Apple says about it:

SceneKit chooses an actual frame rate that is as close as possible to your preferred frame rate based on the capabilities of the screen the view is displayed on. The actual frame rate is usually a factor of the maximum refresh rate of the screen to provide a consistent frame rate.

For example, if the maximum refresh rate of the screen is 60 fps, that is also the highest frame rate the view sets as the actual frame rate. However, if you ask for a lower frame rate, SceneKit might choose 30, 20, 15 or some other factor to be the actual frame rate. For this reason, you want to choose a frame rate that your app can consistently maintain. The default value is 60 fps.

How to lower a View's frame rate to 30 fps?

like image 660
Andy Jazz Avatar asked Dec 01 '18 13:12

Andy Jazz


People also ask

What is the best frame rate for slow-motion animation?

Save your project. As the standard frame rate in Animotica set to 30 FPS, it’s the best fit for your slow-motion. 120 FPS / slowed 4 times = makes perfect 30 FPS. Important! When you slow down a video, the voice slows down accordingly! If you want to film a video blog with voice, 30 FPS will do the best option for you.

What does 24 fps mean in animation?

For example, if your video is 24 FPS, it means there are 24 images in 1 second of footage to produce an animated material. What are the most popular frame rates (FPS)? Most of the Hollywood movies filmed at 24 FPS. It’s one of the most common frame rates on the globe.

What does 60 fps mean in video editing?

In this case, the 60 FPS videos look “sharper” than 24 FPS or 30 FPS ones, because you are recording more pictures per second to describe the movement in a given scene. 60 FPS is also used (along with higher frame rates, such as 120 FPS, 240 FPS) to create smooth slow-motion effects . How can you use FPS in different scenarios?

How to add 120 fps to animotica?

Add your video with 120 FPS to Animotica. Step 2: Apply Slow Motion. Select the uploaded clip and tap on “Speed” button. Change the speed by moving the slider to 0.25x. This was you just slow-downed your 120 FPS video 4 times! Step 3: Select the Right FPS. After editing your video, click on the “Export Video” button.


1 Answers

I solved the issue. SceneKit doesn't allow change frame rate at the moment – it has a fixed 120 fps. So I used SpriteKit framework instead – it allows me use 30 fps.

enter image description here

Here are the changes in my code:

import SpriteKit

class ViewController: UIViewController, ARSKViewDelegate {

    @IBOutlet weak var sceneView: ARSKView!
    // ....................

    override func viewDidLoad() {
        super.viewDidLoad()
        sceneView.delegate = self
        sceneView.preferredFramesPerSecond = 30     // NOW IT WORKS 

        let scene = SKScene()
        sceneView.presentScene(scene)
    }
    // ..................
}
like image 163
Andy Jazz Avatar answered Sep 20 '22 07:09

Andy Jazz