I am trying to get a RGB value in a SceneKit geometry modifier and use that value for positioning my geometry. Here is what I have in my shader.
#include <metal_stdlib>
#include <simd/simd.h>
#include <metal_texture>
#pragma arguments
texture2d terrainRGB;
#pragma transparent
#pragma body
constexpr sampler textureSampler(coord::normalized, address::clamp_to_edge, filter::nearest, mip_filter::nearest);
float4 elevation = terrainRGB.sample(textureSampler, _geometry.texcoords[kSCNTexcoordCount-1]);
float r = elevation.r * 256;
float g = elevation.g * 256;
float b = elevation.b * 256;
float rgbExpression = (r * 256.0 * 256.0) + (g * 256.0) + b;
float e = -10000.0 + (rgbExpression * 0.1);
_geometry.position.z = e;
Then I set up my geometry like this.
let geo = geo = SCNGeometry(...)
let texture = SCNMaterialProperty(contents: img)
geo.setValue(texture, forKeyPath: "terrainRGB")
If I encode my elevations into the geometry normals use the normals instead of a texture to get the elevations this code works. But I don't want to use the normals because that changes how light sources work with my scene.
What I can't figure out is what data type I should pass to geo.setValue()
to work with texture2d
in my shader. Or is there something else I should be doing with a geometry modified to pass a texture and be able to sample it in the modifier.
It turns out there were three things that I needed to do to get this work.
MTLTexture
to SCNMaterialProperty
instead of a UIImage
.SCNMaterialProperty
to match my filters on the sampler
in my geometry modifier. (I am not sure if other combinations would work, this is just what I had to do with my case.)CGColorSpace.genericRGBLinear
. My image in the sRGB color space. I am not exactly sure why I had to do this. I have a vertex shader set up to do the exact same thing with RealityKit and it works just fine with the sRGB image.Here is what my final code to set the texture on my SCNGeometry
looks like.
if let cgImg = image.cgImage,
let colorSpace = CGColorSpace(name: CGColorSpace.genericRGBLinear),
let textureImg = cgImg.copy(colorSpace: colorSpace),
let d = self.sceneDevice { //an MTLDevice
do {
let textureLoader = MTKTextureLoader(device: d)
let texture = try textureLoader.newTexture(cgImage: textureImg)
let scnTexture = SCNMaterialProperty(contents: texture)
scnTexture.minificationFilter = .nearest
scnTexture.magnificationFilter = .nearest
scnTexture.wrapS = .clamp
scnTexture.wrapT = .clamp
geo.setValue(scnTexture, forKeyPath: "terrainRGB")
} catch {
print("Error: \(error)")
}
}