With iOS 18 RealityView is so convenient replacing SceneKit creating 3d But when it comes to moving 3d Entities using 2d touch and drag I face a problem, I could not make an accurate move which like attaching the 3d Entity to my finger while drag it on the screen, it was much easier in SceneKit using project and unproject and Also in ARKit/ARView using raycast also it is very successful in VisionOS cause it is 3D Touch and I can use 3d location in gesture value but in 2d it simply not working
here is the closest I get to and it do move the 3d entity but not responding well to the location of the finger.
import SwiftUI
import RealityKit
struct ContentView: View {
@State var box = Entity()
@State var lastPanTouchPosition: CGPoint = .zero
var body: some View {
RealityView{ content in
let item = ModelEntity(mesh: .generateBox(size: .init(0.25, 0.25, 0.25)), materials: [SimpleMaterial(color: .blue, isMetallic: true)])
box.addChild(item)
content.add(box)
}
.gesture(dragThis)
}
var dragThis: some Gesture {
DragGesture()
.onChanged { value in
print(value.translation)
print(value.location)
let dragRatio: Float = 0.0001
box.position.x += Float(value.translation.width) * dragRatio
box.position.y -= Float(value.translation.height) * dragRatio
}
.onEnded{_ in
box.position = [0, 0, 0]
}
}
}
The beauty of moving AR/VR models in 3D space of iOS RealityView
using a standard 2D gesture is that you don't need to turn on a CollisionComponent
, which means that heavy models are much more responsive. The disadvantage is also obvious: you need to find a balance between the speed of moving your finger on the screen and the speed of model's movement, which may be located at a considerable distance along the Z axis from the camera. This code works well (tested on 12.9" iPad):
import SwiftUI
import RealityKit
struct ContentView : View {
@State var position: UnitPoint = .zero
let box = ModelEntity(mesh: .generateBox(size: 0.25))
let ratio: Float = 0.002
let group = Entity()
var drag: some Gesture {
DragGesture(minimumDistance: 15, coordinateSpace: .global)
.onChanged {
group.position.x = Float($0.translation.width + position.x) * ratio
group.position.y = Float($0.translation.height + position.y) * -ratio
}
.onEnded {
position.x += $0.translation.width
position.y += $0.translation.height
}
}
var body: some View {
RealityView { rvc in
group.addChild(box)
rvc.add(group)
}
.gesture(drag)
}
}