Three.js FirstPersonControls camera rotation jumps back after GSAP tween / restart controls

Trying to rotate camera after mousedown on raycaster with slerp. But after the animation when i turn the control back on, the camera jumps back to its previous rotation. Updating controls seems not to solve this. What i'm missing here?

function mousedown( event ) {
    
  mouse.x = ( event.clientX / windowWidth ) * 2 - 1;
  mouse.y = - ( event.clientY / windowHeight ) * 2 + 1;
  raycaster.setFromCamera( mouse, camera );
  intersects = raycaster.intersectObjects( targets );

  if ( intersects.length > 0 ) {

    // stop controls
    controls.enabled = false;

    var targetOrientation = intersects[0].object.quaternion.normalize();
    gsap.to({}, 1, { 
      onUpdate: function() {
        controls.update();
        //camera.lookAt( intersects[0].object.position );
        camera.quaternion.slerp( targetOrientation , this.progress() );
      },
      onComplete: function() {
        controls.update();
        //camera.lookAt(new THREE.Vector3( 0, 0, 0 ) );

        // restart controls
        controls.enabled = true;
      }
    });
  }
}

https://jsfiddle.net/pixeldino/s30pbzj7/152/

Updated Solution:
https://jsfiddle.net/pixeldino/k2hmbqr3/61/

FirstPersonControls maintains an internal state of the camera's orientation. By using camera.quaternion.slerp() this state gets out of sync.

You can solve this by using FirstPersonControls.lookAt() at the end of your animation. In this way, the internal state will be updated by forcing the controls to look at the destination point.

https://jsfiddle.net/fnw8643a/