RealityKit's `.allowsCameraControl` tool for building `nonAR` apps

RealityKit has an option for .nonAR camera, but lacks fundamental options to build 3D nonAR apps, like camera control (.allowsCameraComtrol).

Is it viable to use RealityKit a replacement for SceneKit (which hasn't been updated for the past years and will likely be desecrated soon)?

realityViewCameraControls(_:) for RealityView

You can implement this feature when working with RealityView in iOS 18+ and macOS 15+.

@MainActor 
@preconcurrency
func realityViewCameraControls(_ controls: CameraControls) -> some View

.allowsCameraControl equivalent for ARView

Apple has been preparing a replacement for SceneKit, and it's definitely a Reality family – RealityKit, RealityFoundation, Reality Composer (iOS/macOS), Reality Composer Pro (visionOS), Reality Converter, ARQuickLook, etc. Using RealityKit 3.0 / 2.0 / 1.0, you can create both AR and VR apps. As a tool for prototyping 3D scenes, you need to use Reality Composer or Reality Composer Pro. And even despite the lack of such an important feature's equivalent as .allowsCameraControl, you're capable of making VR apps using RealityKit.

Workaround for RealityKit's ARView

Use my code as a starting point to create your own camera control for VR in RealityKit's ARView:

import UIKit
import RealityKit

class ViewController: UIViewController {
    @IBOutlet var arView: ARView!
    let buildings = try! Experience.loadScene()
    let camera = PerspectiveCamera()
    var current_X_Angle: Float = 0.0
    var current_Y_Angle: Float = 0.0

    override func viewDidLoad() {
        super.viewDidLoad()
        arView.environment.background = .color(.systemCyan)
        arView.scene.anchors.append(buildings)
        self.camera.position.z = 1
        buildings.children[0].addChild(self.camera)
        self.gestureRecognizer()
    }
    func gestureRecognizer() {
        for gestureRecognizer in [UIPanGestureRecognizer.self,
                                  UIPinchGestureRecognizer.self] {
            if gestureRecognizer == UIPinchGestureRecognizer.self {
                let r = UIPinchGestureRecognizer(target: self,
                               action: #selector(allowCameraControl_01))
                arView.addGestureRecognizer(r)
            }
            if gestureRecognizer == UIPanGestureRecognizer.self {
                let r = UIPanGestureRecognizer(target: self,
                               action: #selector(allowCameraControl_02))
                arView.addGestureRecognizer(r)
            }
        }
    }
    @objc func allowCameraControl_01(recognizer: UIPinchGestureRecognizer) {
        switch recognizer.state {
            case .changed, .ended:
                self.camera.position.z *= 1 / Float(recognizer.scale)
                recognizer.scale = 1.0
            default: break
        }
    }
    @objc func allowCameraControl_02(recognizer: UIPanGestureRecognizer) {
        switch recognizer.state {
            case .changed, .ended:
                let translate = recognizer.translation(in: recognizer.view)
                let angle_X = Float(translate.y / 300) * .pi / 180.0
                let angle_Y = Float(translate.x / 100) * .pi / 180.0
                self.current_X_Angle += angle_X
                self.current_Y_Angle += angle_Y                
                camera.setOrientation(Transform(pitch: current_X_Angle,
                                                  yaw: current_Y_Angle,
                                                 roll: .zero).rotation,
                                      relativeTo: buildings.anchor)
            default: break
        }
    }
}

Here you can find SwiftUI version.

visionOS Xcode Simulator

In Xcode 15 / 16 visionOS simulator, there are Pan, Orbit and Dolly UI buttons, available in lower right corner.