Having successfully managed to get my app running with RealityKit instead of SceneKit, I'm noticing a distinctly lower frame rate with the same model. If anything, the model should be slightly better when it comes to the number of vertices/indices and triangles.
I note that even with SceneKit, if the camera was placed in the model such that a lot of the model was in front of the camera, even if 90% was invisible or obscured by nodes only a short distance in front, the frame rate would drop. I tried reducing the zNear/far settings to help with that but not much luck.
On RealityKit this seems to be much worse. In SceneKit I was seeing on an iPhoneSE20, a reasonably solid 50 to 60FPS. The same model in RealityKit is lucky to see anything more than 20FPS unless I place the camera at an edge and face it outward.
I've seen odd posts here and there that talk about limits of 100K polygons. Well with RealityKit under SwiftUI, I cannot work out how to get the debug frame on screen to see how to measure that (ARView has a debug frame, but RealityView does not that I can find), so I have no idea about polygon counts.
Whilst Xcode fails to capture the view hierarchy on macOS (it times out after 60 seconds), it succeeds if I do so from my iPhoneSE20. The stats that I'm seeing there for the game level are:
Apart from a couple of the textures, everything is programmatically generated. Most of the walls, floors, ceilings in the level are as low in vertices/triangles as I can get them. I also use statically generated Meshes as much as I can to try and reduce the memory footprint.
There are a few places where I have used the RealityKit built-ins to provide a sphere or cylinder, and those tend to have a LOT of triangles, but everything else is very low count. I'm puzzled by the high number of triangles to be honest, but I don't know how to reduce it (yet).
What I am wondering is what I should do. I've got my camera set up as follows:
var cameraComponent = PerspectiveCameraComponent()
cameraComponent.far = WallUtilities.wallWidth * 22.0 // wallWidth = 2.2
cameraComponent.near = 0.1
cameraComponent.fieldOfViewInDegrees = 60.0
the distance being large is to allow for long hallways. Why does RealityKit/SceneKit spend time, presumably in the CPU, processing content that is obscured and out of frame?
Should I set up a RealityKit system to reduce the cameraComponent.far according to the actual visible distance in front of the camera? I would have thought a complex framework like RealityKit/SceneKit would automatically do that.
If you're planning to achieve a smooth rendering in RealityKit iOS app, 100K triangular shaded/textured polygons per scene/asset is a safe / robust maximum. Thus, to maintain an ideal 60 fps refresh rate, the amount of 100K/or less polygons will provide a rendering frame time under 16.67 ms. In RealityKit visionOS app, however, a maximum is 500K triangles. Watch the Optimize 3D assets for spatial computing (time 02:50) video which provides you with polygon count in immersive (i.e. visionOS) app.
The main question is, does the 3D model need to be prepared in any special way to work with RealityKit? The answer is evident: Yes, it definitely does. 3D asset must be prepared in a specific way: .usdz/.reality format (number of meshes in a usdz file can be around 50), poly count limit, limited quantity of PBR shaders, limited quantity of textures with base of 2 resolution, limited number of lights with shadows, etc. As we can see from your data, you have exceeded the safe number of polygons by 8.5 times. First, try reducing the number of polys of .usdz asset in Maya or Blender. Second, try disabling lights/shadows and all PBR shaders.
RealityKit's near and far clipping planes significantly impact rendering. These parameters define the range of distance (frustum volume) from the camera within which objects are visible and rendered. Objects outside this range are clipped, meaning they are not drawn, which directly affects which parts of the AR/VR scene are visible and how much processing power is needed for rendering.