Access Mesh Data in RealityKit

I'm attempting to access the geometry from a ModelEntity loaded from a .usdz file. I've been able to get to the various AnyMeshBuffers for normals, positions, etc using the following code:

let meshModels = modelEntity.model!.mesh.contents.models
for model in meshModels {
    for part in parts {
        for (identifier, buffer) in part.buffers {
            
        }
    }
}

Inside the loop I can print out the AnyMeshBuffer in the debugger and see something like this:

▿ AnyMeshBuffer
  ▿ buffer : BufferEntry<FloatVector3Packed>
    ▿ id : "primvars:normals"
      - name : "primvars:normals"
      - isCustom : true
    - count : 10968
    - rate : RealityFoundation.MeshBuffers.Rate.vertex
    - elementType : RealityFoundation.MeshBuffers.ElementType.simd3Float
    - packed : true
    ▿ buffer : MeshBuffer<FloatVector3Packed>
      ▿ closure : Closures
        - getArray : (Function)
        - getIndices : (Function)
        - getData : (Function)
        - chunk : (Function)
      - elementType : RealityFoundation.MeshBuffers.ElementType.simd3Float
      - packed : true
      - count : 10968
      - rate : RealityFoundation.MeshBuffers.Rate.vertex

However, I haven't been able to actually go any further. There's a get function in AnyMeshBuffer that returns a MeshBuffer but I haven't been able figure out what type to use.

Does anyone have an example showing how to access the actual geometry from a ModelEntity?

Solution

Short answer

You can access the mesh elements directly from each part in your example:

for part in parts {
   let vertices = part.positions.elements
   let normals = part.normals?.elements
   // etc.
}

This is because MeshResource.Part conforms to MeshBufferContainer

Accessing Mesh Data in RealityKit

Here’s an example of how to access the mesh geometry data from a MeshResource by extending the ModelEntity class. If the ModelEntity contains a ModelComponent, the function iterates through each mesh model (MeshResource.Model) within that component. It then goes through each part (MeshResource.Part) of these mesh models to access their respective geometry data.

extension ModelEntity {
    func getMeshGeometry() {
        // ModelComponent of the ModelEntity
        if let modelComponent = self.model {
            // For MeshResource.Model in MeshModelCollection 
            for meshModel in modelComponent.mesh.contents.models {
                // For MeshResource.Part in MeshPartCollection 
                for part in meshModel.parts {
                    let vertices: [SIMD3<Float>] = part.positions.elements
                    let indices: [UInt32]? = part.triangleIndices?.elements
                    let texture: [SIMD2<Float>]? = part.textureCoordinates?.elements
                    let normals: [SIMD3<Float>]? = part.normals?.elements
                    let tangents: [SIMD3<Float>]? = part.tangents?.elements
                    
                    // Convenience Print 
                    print("Model:\(meshModel.id).Part:\(part.id)(Vertices:\(vertices.count), Indices:\(indices?.count), TextureCoordinates:\(texture?.count), Normals: \(normals?.count), Tangents: \(tangents?.count))")
                }
            }
        }
    }
}

This also works for ModelEntity entities created from USDZ models

Documentation