The set up:
I am developing a visionOS app that uses an immersive space.
The user sees a board with entities put onto it.
My app places the board in front of the default camera and entities with a certain position and orientation relative to the board. Placement and rotation should be animated.
The problem:
If I place the entities by assigning a Transform
to the transform
property of the entity directly, i.e. without animation, the result is correct.
However I have to use the entity's move(to:
function to animate it. And move(to:
works in an unexpected way.
I thus wrote a little test app, based on Apple's visionOS immersive app template (below). There, the following 5 cases are treated:
Set transform
directly (without animation). This gives the correct result, and works as expected (without animation)
Set transform
using move
relative to world (without animation). This gives the correct result, although it does not work as expected. I expected "relative to world" means translation and rotation is relativ to world. This seems wrong for translation and right for rotation.
Set transform
using move
relative to parentEntity
(without animation). This gives a wrong result, although translation
and rotation
are defined relative to the parentEntity
.
Set transform
using move
relative to world with animation. This gives also a wrong result, and without animation.
Set transform
using move
relative to parentEntity
with animation. This gives also a wrong result, and without animation.
Here are the screen shots for the cases 1...5:
The question:
So, obviously, I don't understand what move(to:
does. I would be happy to get any advice what is wrong and how to do it right.
Here is the code:
import SwiftUI
import RealityKit
import RealityKitContent
struct ImmersiveView: View {
@Environment(AppModel.self) var appModel
let boardHeight: Float = 0.1
let boxHeight: Float = 0.3
var body: some View {
RealityView { content in
let boardEntity = makeBoard()
content.add(boardEntity)
let boxEntity = makeBox(parentEntity: boardEntity)
boardEntity.addChild(boxEntity)
}
}
func makeBoard() -> ModelEntity {
let mesh = MeshResource.generateBox(width: 1.0, height: boardHeight, depth: 1.0)
var material = UnlitMaterial(); material.color.tint = .red
let boardEntity = ModelEntity(mesh: mesh, materials: [material])
boardEntity.transform.translation = [0, 0, -3]
return boardEntity
}
func makeBox(parentEntity: Entity) -> ModelEntity {
let mesh = MeshResource.generateBox(width: 0.3, height: boxHeight, depth: 0.3)
var material = UnlitMaterial(); material.color.tint = .green
let boxEntity = ModelEntity(mesh: mesh, materials: [material])
// Set position and orientation of the box
// To put the box onto the board, move it up by half height of the board and half height of the box
let y_up = boardHeight/2.0 + boxHeight/2.0
let translation = SIMD3<Float>(0, y_up, 0)
// Turn the box by 45 degrees around the y axis
let rotationY = simd_quatf(angle: Float(45.0 * .pi/180.0), axis: SIMD3(x: 0, y: 1, z: 0))
let transform = Transform(rotation: rotationY, translation: translation)
// Do the actual move
// 1) Set transform directly (without animation)
boxEntity.transform = transform // Translation and rotation correct
// 2) Set transform using move relative to world (without animation)
// boxEntity.move(to: transform, relativeTo: nil) // Translation and rotation correct
// 3) Set transform using move relative to parentEntity (without animation)
// boxEntity.move(to: transform, relativeTo: parentEntity) // Translation incorrect, rotation correct
// 4) Set transform using move relative to world with animation
// boxEntity.move(to: transform,
// relativeTo: nil,
// duration: 1.0,
// timingFunction: .linear) // Translation incorrect, rotation incorrect, no animation
// 5) Set transform using move relative to parentEntity with animation
// boxEntity.move(to: transform,
// relativeTo: parentEntity,
// duration: 1.0,
// timingFunction: .linear) // 5) Translation incorrect, rotation incorrect, no animation
return boxEntity
}
}
This seems to be a RealityKit bug.
I posted the question on Apple's developer forum, and a Vision Pro Engineer answered and found a workaround (abbreviated):
This may be a bug, but our teams will need to further investigate. Please consider filing a bug using Feedback Assistant.
I did so (FB15152333).
Here is the workaround:
import SwiftUI
import RealityKit
import RealityKitContent
struct ImmersiveView: View {
@Environment(AppModel.self) var appModel
let boardHeight: Float = 0.1
let boxHeight: Float = 0.3
var body: some View {
RealityView { content in
let boardEntity = makeBoard()
content.add(boardEntity)
let mesh = MeshResource.generateBox(width: 0.3, height: boxHeight, depth: 0.3)
var material = UnlitMaterial(); material.color.tint = .green
let boxEntity = ModelEntity(mesh: mesh, materials: [material])
boardEntity.addChild(boxEntity)
Task {
moveBox(boxEntity, parentEntity: boardEntity)
}
}
}
func makeBoard() -> ModelEntity {
let mesh = MeshResource.generateBox(width: 1.0, height: boardHeight, depth: 1.0)
var material = UnlitMaterial(); material.color.tint = .red
let boardEntity = ModelEntity(mesh: mesh, materials: [material])
boardEntity.transform.translation = [0, 0, -3]
return boardEntity
}
func moveBox(_ boxEntity:Entity, parentEntity: Entity) {
// Set position and orientation of the box
// To put the box onto the board, move it up by half height of the board and half height of the box
let y_up = boardHeight/2.0 + boxHeight/2.0
let translation = SIMD3<Float>(0, y_up, 0)
// Turn the box by 45 degrees around the y axis
let rotationY = simd_quatf(angle: Float(45.0 * .pi/180.0), axis: SIMD3(x: 0, y: 1, z: 0))
let transform = Transform(rotation: rotationY, translation: translation)
// Do the actual move
// 5) Set transform using move relative to parentEntity with animation
boxEntity.move(to: transform,
relativeTo: parentEntity,
duration: 1.0,
timingFunction: .linear) // 5) Translation incorrect, rotation incorrect, no animation
}
}