Yet another RealityKit question.

I have a player Entity whose movements throughout the world model I want to be able to script. I've managed to translate the simple SceneKit actions to RealityKit, but have a problem in one area (so far).

When the player turns a corner and then heads down a path, I group two actions, one to turn the player 90 degrees, and the other to move the player forward.

By grouping the actions, the turn is executed with a small duration so that it is executed as the player starts moving, and then the movement continues.

The code below is my rough attempt to make this work with RealityKit. What I do is take a copy of the player transform as at the start of the turn/movement for each of the two actions (newMoveTransform and newTurnTransform), and then adjust the appropriate property of each.

The idea is to run the two actions in parallel as a group.

// our group of actions
var groupActions : [AnimationResource] = []

// grab the current player transform.
var newMoveTransform = currentPlayerTransform
var newTurnTransform = currentPlayerTransform

// adjust the translation
newMoveTransform.translation = newMoveTransform.translation + distanceAsVector

// adjust the rotation
newTurnTransform.rotation = simd_quatf(angle: newDirection, axis: SIMD3<Float>(0.0, 1.0, 0.0))

// add the movement action
groupActions.append(try AnimationResource.makeActionAnimation(for: FromToByAction(from: currentPlayerTransform, to: newMoveTransform),
                                                              duration: 10.0, bindTarget: .transform))

// add the turn action.
groupActions.append(try AnimationResource.makeActionAnimation(for: FromToByAction(from: currentPlayerTransform, to: newTurnTransform),
                                                              duration: 2.0, bindTarget: .transform))

// now copy the target rotation into the move transform so that we can save it to
// the "current" transform.
newMoveTransform.rotation = newTurnTransform.rotation

// update the current transform so that we can use it for the next step in the path.
currentPlayerTransform = newMoveTransform

what I'm seeing however is that as soon as the turn completes, the movement action reverts the rotation to the value it has prior to the turn so that the remainder of the movement becomes a sideways strafing movement.

UPDATE:

In writing this out I had an idea. It's a bit ugly but it seems to work. If I change the above to be:

// our group of actions
var groupActions : [AnimationResource] = []

// grab the current player transform.
var newMoveTransform = currentPlayerTransform
var newTurnTransform = currentPlayerTransform

// adjust the translation
newMoveTransform.translation = newMoveTransform.translation + distanceAsVector

// adjust the rotation
newTurnTransform.rotation = simd_quatf(angle: newDirection, axis: SIMD3<Float>(0.0, 1.0, 0.0))

//  now copy the target rotation to the move transform...
newMoveTransform.rotation = newTurnTransform.rotation

// and now create a starting point for the move action where the rotation is already
// complete and use that in the move action.
var moveFromTransform = currentPlayerTransform
moveFromTransform.rotation = newTurnTransform.rotation
                        
groupActions.append(try AnimationResource.makeActionAnimation(for: FromToByAction(from: moveFromTransform, to: newMoveTransform),
                                                                                      duration: 10.0, bindTarget: .transform))

// add the turn action.
groupActions.append(try AnimationResource.makeActionAnimation(for: FromToByAction(from: currentPlayerTransform, to: newTurnTransform),
                                                              duration: 2.0, bindTarget: .transform))

// now copy the target rotation into the move transform so that we can save it to
// the "current" transform.
newMoveTransform.rotation = newTurnTransform.rotation

// update the current transform so that we can use it for the next step in the path.
currentPlayerTransform = newMoveTransform

the main change here is that now the movement action is using the target rotation all the way. for whatever reason, this does not override the turn animation in the second action, so visually it looks OK.

I just don't quite understand why this works as well as it does, and would love a simpler, more elegant way of doing this.

As it stands, to build up a sequence of actions that move an entity from one place to another, I basically have to keep track of where the entity should be as I build the sequence and keep updating it (i.e. the currentPlayerTransform above) as I build the sequence. With SceneKit, none of that was needed as the SCNActions could be created and sequenced with the knowledge that when they ran, they would act on the Entity/Node as it was at that time.

I did not manage to find a solution without help. As I mentioned in another question, I happened upon a package on GitHub called RealityActions that solved a lot of problems with animation in RealityKit.

It most definitely solved the issues I am asking about in this question. My solution for this is:

func turnAndMoveAction(byAngle angle: Float, andDistance distanceAsVector: SIMD3<Float>, withDuration duration: TimeInterval) -> FiniteTimeAction {
    return Group([
        MoveBy(duration: duration, delta: distanceAsVector),
        RotateBy(duration: min(duration, PlayerNode.animationDuration), deltaAnglesRad: SIMD3<Float>(0.0, angle, 0.0))
    ])
}

func demo() {
    self.start(turnAndMoveAction(byAngle: .pi / 2.0, andDistance: SIMD3<Float>(12.0, 0.0, 0.0), withDuration: 3.0))
}

this runs the grouped actions exactly as I had expected, without them interfering with each other, and with them running for the duration one would expect.

I cannot recommend RealityActions enough if you are coming from SceneKit, SpriteKit or Cocos2D.