swiftrealitykitvisionosimmersive-space

Don't understand RealityKit's entity.move in visionOS immersive space


The set up:
I am developing a visionOS app that uses an immersive space.
The user sees a board with entities put onto it. My app places the board in front of the default camera and entities with a certain position and orientation relative to the board. Placement and rotation should be animated.

The problem:
If I place the entities by assigning a Transform to the transform property of the entity directly, i.e. without animation, the result is correct.
However I have to use the entity's move(to: function to animate it. And move(to: works in an unexpected way.
I thus wrote a little test app, based on Apple's visionOS immersive app template (below). There, the following 5 cases are treated:

  1. Set transform directly (without animation). This gives the correct result, and works as expected (without animation)

  2. Set transform using move relative to world (without animation). This gives the correct result, although it does not work as expected. I expected "relative to world" means translation and rotation is relativ to world. This seems wrong for translation and right for rotation.

  3. Set transform using move relative to parentEntity (without animation). This gives a wrong result, although translation and rotation are defined relative to the parentEntity.

  4. Set transform using move relative to world with animation. This gives also a wrong result, and without animation.

  5. Set transform using move relative to parentEntity with animation. This gives also a wrong result, and without animation.

Here are the screen shots for the cases 1...5:

enter image description here
Cases 1 & 2

enter image description here
Case 3

enter image description here
Cases 4 & 5

The question:
So, obviously, I don't understand what move(to: does. I would be happy to get any advice what is wrong and how to do it right.

Here is the code:

import SwiftUI
import RealityKit
import RealityKitContent

struct ImmersiveView: View {
    @Environment(AppModel.self) var appModel
    
    let boardHeight: Float = 0.1
    let boxHeight: Float = 0.3

    var body: some View {
        RealityView { content in
            let boardEntity = makeBoard()
            content.add(boardEntity)
            let boxEntity = makeBox(parentEntity: boardEntity)
            boardEntity.addChild(boxEntity)
        }
    }
    
    func makeBoard() -> ModelEntity {
        let mesh = MeshResource.generateBox(width: 1.0, height: boardHeight, depth: 1.0)
        var material = UnlitMaterial(); material.color.tint = .red
        let boardEntity = ModelEntity(mesh: mesh, materials: [material])
        boardEntity.transform.translation = [0, 0, -3]
        return boardEntity
    }
    
    func makeBox(parentEntity: Entity) -> ModelEntity {
        let mesh = MeshResource.generateBox(width: 0.3, height: boxHeight, depth: 0.3)
        var material = UnlitMaterial(); material.color.tint = .green
        let boxEntity = ModelEntity(mesh: mesh, materials: [material])
        
        // Set position and orientation of the box
        
        // To put the box onto the board, move it up by half height of the board and half height of the box
        let y_up = boardHeight/2.0 + boxHeight/2.0
        let translation = SIMD3<Float>(0, y_up, 0)
        // Turn the box by 45 degrees around the y axis
        let rotationY = simd_quatf(angle: Float(45.0 * .pi/180.0), axis: SIMD3(x: 0, y: 1, z: 0))
        let transform = Transform(rotation: rotationY, translation: translation)
        
        // Do the actual move
        
        // 1) Set transform directly (without animation)
        boxEntity.transform = transform // Translation and rotation correct
        
        // 2) Set transform using move relative to world (without animation)
//      boxEntity.move(to: transform, relativeTo: nil) // Translation and rotation correct
        
        // 3) Set transform using move relative to parentEntity (without animation)
//      boxEntity.move(to: transform, relativeTo: parentEntity) // Translation incorrect, rotation correct
        
        // 4) Set transform using move relative to world with animation
//      boxEntity.move(to: transform,
//                     relativeTo: nil,
//                     duration: 1.0,
//                     timingFunction: .linear) // Translation incorrect, rotation incorrect, no animation
        
        // 5) Set transform using move relative to parentEntity with animation
//      boxEntity.move(to: transform,
//                     relativeTo: parentEntity,
//                     duration: 1.0,
//                     timingFunction: .linear) // 5) Translation incorrect, rotation incorrect, no animation

        return boxEntity
    }

}

Solution

  • This seems to be a RealityKit bug.
    I posted the question on Apple's developer forum, and a Vision Pro Engineer answered and found a workaround (abbreviated):

    This may be a bug, but our teams will need to further investigate. Please consider filing a bug using Feedback Assistant.

    I did so (FB15152333).

    Here is the workaround:

    import SwiftUI
    import RealityKit
    import RealityKitContent
    
    struct ImmersiveView: View {
        @Environment(AppModel.self) var appModel
        
        let boardHeight: Float = 0.1
        let boxHeight: Float = 0.3
        
        var body: some View {
            RealityView { content in
                let boardEntity = makeBoard()
                content.add(boardEntity)
                let mesh = MeshResource.generateBox(width: 0.3, height: boxHeight, depth: 0.3)
                var material = UnlitMaterial(); material.color.tint = .green
                let boxEntity = ModelEntity(mesh: mesh, materials: [material])
                boardEntity.addChild(boxEntity)
                
                Task {
                    moveBox(boxEntity, parentEntity: boardEntity)
                }
            }
        }
        
        func makeBoard() -> ModelEntity {
            let mesh = MeshResource.generateBox(width: 1.0, height: boardHeight, depth: 1.0)
            var material = UnlitMaterial(); material.color.tint = .red
            let boardEntity = ModelEntity(mesh: mesh, materials: [material])
            boardEntity.transform.translation = [0, 0, -3]
            return boardEntity
        }
        
        func moveBox(_ boxEntity:Entity, parentEntity: Entity)  {
            // Set position and orientation of the box
            
            // To put the box onto the board, move it up by half height of the board and half height of the box
            let y_up = boardHeight/2.0 + boxHeight/2.0
            let translation = SIMD3<Float>(0, y_up, 0)
            // Turn the box by 45 degrees around the y axis
            let rotationY = simd_quatf(angle: Float(45.0 * .pi/180.0), axis: SIMD3(x: 0, y: 1, z: 0))
            let transform = Transform(rotation: rotationY, translation: translation)
            
            // Do the actual move
            
            // 5) Set transform using move relative to parentEntity with animation
            boxEntity.move(to: transform,
                           relativeTo: parentEntity,
                           duration: 1.0,
                           timingFunction: .linear) // 5) Translation incorrect, rotation incorrect, no animation
        }
        
    }