iosswiftsprite-kitscenekit360-virtual-reality

Wrong camera position playing 360 degree video, using SceneKit and CoreMotion


I am playing around with a 360 degree video player using SpriteKit, SceneKit and CoreMotion. The player is working so far, but the video is always zoomed-in a bit. It looks like the camera position is not at (0,0,0), but somehow wrong on the Z axis. Unfortunately, I have not found a way to adjust that. To reproduce the behavior just tap the screen, when the video is playing and pinch-zoom-out. This enables camera control with gestures, double tapping returns to camera control with the device.

import UIKit
import SceneKit
import CoreMotion
import SpriteKit
import AVFoundation

class Video360VC: UIViewController {
let motionManager = CMMotionManager()
let cameraNode = SCNNode()
@IBOutlet weak var sceneView: SCNView!

@IBAction func exitBtnClicked(_ sender: Any) {
performSegueToReturnBack()
}

func createSphereNode(material: AnyObject?) -> SCNNode {
    let sphere = SCNSphere(radius: 20.0)
    sphere.firstMaterial!.isDoubleSided = true
    sphere.firstMaterial!.diffuse.contents = material
    let sphereNode = SCNNode(geometry: sphere)
    sphereNode.position = SCNVector3Make(0,0,0)
    sphereNode.rotation = SCNVector4Make(1, 0, 0, Float.pi)
    return sphereNode
}

func configureScene(node sphereNode: SCNNode) {
    // Set the scene
    let scene = SCNScene()
    sceneView.scene = scene
    sceneView.showsStatistics = true
    sceneView.allowsCameraControl = true

    // Camera, ...
    cameraNode.camera = SCNCamera()
    cameraNode.position = SCNVector3Make(0, 0, 0)
    scene.rootNode.addChildNode(sphereNode)
    scene.rootNode.addChildNode(cameraNode)
}

func startCameraTracking() {
    motionManager.deviceMotionUpdateInterval = 1.0 / 60.0
    motionManager.startDeviceMotionUpdates(to: OperationQueue.main) {
        [weak self](data: CMDeviceMotion?, error: Error?) in
        guard let data = data else { return }
        let attitude: CMAttitude = data.attitude

        self?.cameraNode.eulerAngles = SCNVector3Make(-   Float(attitude.roll + Double.pi/2), Float(attitude.yaw), -Float(attitude.pitch))
    }
}

override func viewDidLoad() {
    super.viewDidLoad()

    guard let fileURL = Bundle.main.url(forResource: "360Test2", withExtension: "mp4") else {
        print("Video File not found")
        return
    }

    let player = AVPlayer(url: fileURL)
    let videoNode = SKVideoNode(avPlayer: player)
    let size = CGSize(width: 1280, height: 720)
    videoNode.size = size
    videoNode.position = CGPoint(x: size.width/2, y: size.height/2)

    let spriteScene = SKScene(size: size)
    spriteScene.scaleMode = .resizeFill
    spriteScene.addChild(videoNode)

    let sphereNode = createSphereNode(material:spriteScene)
    configureScene(node: sphereNode)
    guard motionManager.isDeviceMotionAvailable else {
        fatalError("Device motion is not available")
    }
    startCameraTracking()
    player.play()
}

override func viewWillAppear(_ animated: Bool) {
    self.sceneView.play(self)
}

override func didReceiveMemoryWarning() {
    super.didReceiveMemoryWarning()
}
}

I used this video for testing, which can also be downloaded here. Thanks a lot !


Solution

  • From what i can gather from your explanation is, that you have a problem with the so called Field of View. It may look like the camera is zoomed but actually it is not. Look for the FOV settings in your camera try settings like 90 degree.