swiftcollision-detectionuitapgesturerecognizerrealitykitreality-composer

How to make grouped objects in one Reality Composer scene draggable?


So I have a Reality Composer file "Experience" loaded onto an XCode project with three scenes.

I have added behaviors and actions that work in XCode. The actions work perfectly, hiding and showing the 2 other scenes when I click on the different objects.

But on one scene, named "cubes", I added two cubes. I grouped these two cubes as "twoCubes" whereby, according to Apple's documentation, they will now behave as a single object:

You can combine multiple assets in a scene into a group. Grouped objects behave as a single combined object in Reality Composer’s scene view.

There are no associated actions on this scene, I only wanted to make the two cubes draggable and scalable when I pinch. I haven't been able to make that happen. Even when I name each one of the cubes differently, nothing happens.

I suspect it has something to do with what happens when objects are grouped [reference from Apple documentation here ] but I haven't been able to figure out a solution. I've looked at a lot of answers from resident genius Andy Jazz who is the expert on this subject, but nothing... I am doing this because I have a much larger model I am working on my own application, that has about 20 children per scene... so I am hoping I can figure out a solution here that I can use for something much larger, later...

see the two cubes to the right

screenshot of the Reality Composer scenes

I have tried multiple Reality Composer models where I was able to drag when viewing the reality file on my phone, but that capability was removed once I imported the scenes into an Xcode model.

I have tried everything, including a single cube... and dragging doesn't work either.

import UIKit
import RealityKit

class ViewController: UIViewController {
    
    @IBOutlet var arView: ARView!
    
    override func viewDidLoad() {
        super.viewDidLoad()
        
        let boxAnchor = try! Experience.loadBox()
        arView.scene.anchors.append(boxAnchor)
        
        let cubesAnchor = try! Experience.loadCubes()
        arView.scene.anchors.append(cubesAnchor)
                   
        guard let addingGestures = cubesAnchor.twoCubes as? ModelEntity else { return }
        addingGestures.generateCollisionShapes(recursive: true)
        arView.installGestures([.all], for: addingGestures as (Entity & HasCollision))            
        
        DispatchQueue.main.asyncAfter(deadline: .now() + 2.0){
            self.arView.scene.anchors.removeAll()
            let coneAnchor = try! Experience.loadCone()
            self.arView.scene.anchors.append(coneAnchor)                
            print(coneAnchor)                
        }
        
        func handleTapOnEntity(_ entity: Entity?) {
            guard let entity = entity else { return }  
        }
    }
}

See related question here


Solution

  • What Models Grouping actually means

    In Reality Composer and RealityKit, grouping several objects together means combining several children under single parent (i.e. parenting op). Grouped objects can be simultaneously moved, rotated and scaled using their Parent Entity. However, this cannot be applied to the physics – or in other words, collisions, various dynamics forces, raycasting and three types of RealityKit gestures (EntityGestureRecognizer).

    As you wrote, this is unambiguously stated in the docs.

    If you’ve enabled physics in your scene, grouped objects still behave as separate, individual objects.

    Thus, if you grouped models in Reality Composer, you can't make the two cubes draggable / scalable when you apply corresponding pan or pinch gestures to a group. Nevertheless, there is one workaround.

    Workaround

    In RealityKit, create an empty group as a ModelEntity. Make its CollisionShape larger than both models. In this case, you can use pan, rotate and pinch gestures to control several objects at once. You can generate collision shape as box, sphere, capsule, and convex. You can also create your collision shape from separate parts.

    import UIKit
    import RealityKit
    
    class ViewController: UIViewController {
        @IBOutlet var arView: ARView!
        
        override func viewDidLoad() {
            super.viewDidLoad()
            self.arView.debugOptions = .showPhysics
            
            let sph01 = ModelEntity(mesh: .generateSphere(radius: 0.05))
            sph01.position.x = -0.1
            let sph02 = ModelEntity(mesh: .generateSphere(radius: 0.05))
            sph02.position.x = 0.1
        
            let group = ModelEntity() as ModelEntity & HasCollision
            group.addChild(sph01)
            group.addChild(sph02)
            
            group.generateCollisionShapes(recursive: false)
            self.arView.installGestures(.all, for: group)
            
            let shape = ShapeResource.generateSphere(radius: 0.16)
            let collision = CollisionComponent(shapes: [shape],
                                                 mode: .trigger,
                                               filter: .sensor)    
            group.components.set(collision)
            
            let anchor = AnchorEntity()
            anchor.addChild(group)
            anchor.scale = [5,5,5]
            arView.scene.anchors.append(anchor)    
        }
    }
    

    enter image description here