So I'm a fractal enthusiast and decided to build a 2D/3D fractal generator in WebGL using raymarching, with Typescript as scripting language. I've been a C#/Typescript dev for several years but having zero experience with 3d programming, I used Michael Walczyk's blog as a starting point. Some of my code I use here is derived from his tutorial.
I added the functionality that you can move through the object using WASDQEZC keys. WS = strafe forward-back, AD = strafe left-right, QE = strafe up-down, ZC = roll left-right. I combine this with a mouse look function which moves in the direction the mouse pointer is located on the rendering canvas. So what I want is total freedom of movement like in a spacesim. For this I am using a separate camera rotation matrix together with translation values and send them to the shader like this:
setCameraMatrix(): void {
let cameraRotationMatrixLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_cameraRotation");
let cameraTranslationLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_cameraTranslation");
let foVLocation: WebGLUniformLocation = this.currentContext.getUniformLocation(this.currentProgram, "u_foV");
//add point of camera rotation at beginning
let cameraRotationMatrix: Array<number> = Matrix3D.identity();
//set camera rotation and translation, Z-axis (heading) first, then X-axis (pitch), then Y-axis (roll)
cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateZ(this.cameraRotateZ * Math.PI / 180));
cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateX(this.cameraRotateX * Math.PI / 180));
cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.rotateY(this.cameraRotateY * Math.PI / 180));
//cameraRotationMatrix = Matrix3D.multiply(cameraRotationMatrix, Matrix3D.translate(this.cameraTranslateX, this.cameraTranslateY, this.cameraTranslateZ));
cameraRotationMatrix = Matrix3D.inverse(cameraRotationMatrix);
let cameraPosition: Array<number> = [
this.cameraTranslateX,
this.cameraTranslateY,
-this.cameraTranslateZ,
];
this.currentContext.uniformMatrix4fv(cameraRotationMatrixLocation, false, cameraRotationMatrix);
this.currentContext.uniform3fv(cameraTranslationLocation, cameraPosition);
this.currentContext.uniform1f(foVLocation, this.foV);
}
I tried adding the camera translation values to the camera matrix but that didn't work. I got weird distortion effects and couldn't get it right so I commented that line out and left it there for now for clarity. The reason I did it this way is because of the way my GLSL code is constructed:
The main function from the fragment shader with the call to the ray_march function. v_position is a vec2 with x,y coordinates coming from the vertex shader.:
void main() {
outColor = vec4(ray_march(u_cameraTranslation, u_cameraRotation * vec4(rayDirection(u_foV,v_position),1), u_world, vec3(u_light * vec4(0,0,0,1)).xyz ).xyz,1);
}
The ray_march function I am using. This derives from the sample code in Michael Walczyk's blog.
vec3 ray_march(in vec3 ro, in vec4 rd, in mat4 wm, in vec3 lightPosition) //ro = ray origin, rd = ray direction gt = geometry position after matrix multiplication
{
float total_distance_traveled = 0.0;
const int NUMBER_OF_STEPS = 1024;
float MINIMUM_HIT_DISTANCE = 0.001 * min_hit_distance_correction;
const float MAXIMUM_TRACE_DISTANCE = 1000.0;
for (int i = 0; i < NUMBER_OF_STEPS; i++)
{
vec3 current_position = ro + total_distance_traveled * vec3(rd);
float distance_to_closest = map(current_position, wm);
if (distance_to_closest < MINIMUM_HIT_DISTANCE)
{
vec3 normal = calculate_normal(current_position, wm);
vec3 outColor = vec3(1.0,0,0);
vec3 v_surfaceToLight = lightPosition - current_position;
vec3 v_surfaceToView = ro - current_position;
//insert lighting code below this line
return outColor;
}
if (total_distance_traveled > MAXIMUM_TRACE_DISTANCE)
{
break;
}
total_distance_traveled += distance_to_closest;
}
return vec3(0.25);//gray background
}
The rayDirection function I am using.
vec3 rayDirection(float fieldOfView, vec2 p) {
float z = 1.0 / (tan(radians(fieldOfView) / 2.0));
return normalize(vec3(p.xy, -z));
}
I have issues moving and rotating my camera correctly in 3d world. I'm doing this by applying some trigonometry to get the movement right. E.g., when I move forward, that is the Z-axis. But when I make a 90 degree turn to the right the X-axis now becomes the Z-axis. I am using trigonometry to correct this and actually got something working but now I am stuck in a quagmire of trigonometry with no end in sight and I have a feeling there must be a better and less complicated way. To see what I'm talking about, here is the code of the 'move' function:
move(event: KeyboardEvent): void {
//strafe forward-back
let tXForwardBack: number = (Math.sin(this.cameraRotateY * Math.PI / 180) * Math.cos(this.cameraRotateX * Math.PI / 180)) * this.clipSpaceFactor * this.speed;
let tYForwardBack: number = Math.sin(this.cameraRotateX * Math.PI / 180) * this.speed;
let tZForwardBack: number = (Math.cos(this.cameraRotateY * Math.PI / 180) * Math.cos(this.cameraRotateX * Math.PI / 180)) * this.clipSpaceFactor * this.speed;
//strafe up-down
let tXUpDown: number = ((Math.sin(this.cameraRotateX * Math.PI / 180) * Math.sin(this.cameraRotateY * Math.PI / 180)) * this.clipSpaceFactor * this.speed);
let tYUpDown: number = Math.cos(this.cameraRotateX * Math.PI / 180) * this.speed;
let tZUpDown: number = Math.sin(this.cameraRotateX * Math.PI / 180) * Math.cos(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;
//strafe left-right without roll. TODO: implement roll
let tXLeftRight: number = Math.cos(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;
let tYLeftRight: number = 0;
let tZLeftRight: number = Math.sin(this.cameraRotateY * Math.PI / 180) * this.clipSpaceFactor * this.speed;
switch (event.key) {
case "w": { //strafe forward
this.cameraTranslateX = this.cameraTranslateX + tXForwardBack;
this.cameraTranslateY = this.cameraTranslateY - tYForwardBack;
this.cameraTranslateZ = this.cameraTranslateZ + tZForwardBack;
//this.cameraTranslateZ = this.cameraTranslateZ + (this.clipSpaceFactor * this.speed);
break;
}
case "s": { //strafe back
this.cameraTranslateX = this.cameraTranslateX - tXForwardBack;
this.cameraTranslateY = this.cameraTranslateY + tYForwardBack;
this.cameraTranslateZ = this.cameraTranslateZ - tZForwardBack;
break;
}
case "a": {//strafe left
this.cameraTranslateX = this.cameraTranslateX - tXLeftRight;
this.cameraTranslateY = this.cameraTranslateY + tYLeftRight;
this.cameraTranslateZ = this.cameraTranslateZ + tZLeftRight;
break;
}
case "d": { //strafe right
this.cameraTranslateX = this.cameraTranslateX + tXLeftRight;
this.cameraTranslateY = this.cameraTranslateY - tYLeftRight;
this.cameraTranslateZ = this.cameraTranslateZ - tZLeftRight;
break;
}
case "q": { //strafe up
this.cameraTranslateX = this.cameraTranslateX + tXUpDown;
this.cameraTranslateY = this.cameraTranslateY + tYUpDown;
this.cameraTranslateZ = this.cameraTranslateZ + tZUpDown;
break;
}
case "e": { //strafe down
this.cameraTranslateX = this.cameraTranslateX - tXUpDown;
this.cameraTranslateY = this.cameraTranslateY - tYUpDown;
this.cameraTranslateZ = this.cameraTranslateZ - tZUpDown;
break;
}
case "z": { //roll left
this.cameraRotateZ = (this.cameraRotateZ + (this.sensitivity * this.speed)) % 360;
break;
}
case "c": { //roll right
this.cameraRotateZ = (this.cameraRotateZ - (this.sensitivity * this.speed)) % 360;
break;
}
}
It actually works to some degree, but you can see where this is going :( Also, I get a 'dead' zone when I look up and down along the Y-axis. I found This thread which seems to describe my problem and says 'The trick is to apply the translation to the z-axis but in the local coordinate system of the camera.'
But how do I do that with my existing code? I tried multiplying the world matrix u_world by the rotationmatrix u_rotationMatrix but then the lighting changes as well and it's just an object rotation instead of a separate camera rotation. In the thread I posted there is no lighting so multiplying the camera matrix with the world matrix works for them. But it doesn't for me because of the lighting I implemented. Also, I can't seem to apply the normals separately this way so that I only apply the normals to the world matrix and not to the camera rotation matrix, so that the lighting stays in place when I rotate/translate the camera.
The only way I can get correct normals to the world matrix and a separate cameramatrix is by multiplying the rotationMatrix with the rayDirection like so u_cameraRotation * vec4(rayDirection(u_foV,v_position),1). But when I do this I have to apply all this horrible, partially working trigonometry mess to get something decent. What I want is getting it to work like 'The trick is to apply the translation to the z-axis but in the local coordinate system of the camera.'
But I don't know how. I tried all kinds of things but I'm currently stuck. Any help would be greatly appreciated. I think I've outlined my problem sufficiently enough, if you miss anything please let me know. Thanks in advance.
Looks like I found the answer myself. I applied part of Adisak's answer from this question which is similar to mine. I applied his EulerAnglesToMatrix function with rotation order ZXY, then extracted the x, y and z-axis like so:
let mx: Array<number> = Matrix3D.eulerAnglesToMatrix(pitch, yaw, roll, "ZXY");
let xAxis: Array<number> = mx.slice(0, 3); //x,y,z
let yAxis: Array<number> = mx.slice(3, 6); //x,y,z
let zAxis: Array<number> = mx.slice(6, 9); //x,y,z
I then applied the translation like so, setting the [this.cameraTranslateX,this.cameraTranslateY,this.cameraTranslateZ] as the uniform vec3 u_cameraTranslation variable for the fragmentshader:
switch (event.key) {
case "w": { //strafe forward
this.cameraTranslateX = this.cameraTranslateX - ((zAxis[0]) * this.clipSpaceFactor * this.speed);
this.cameraTranslateY = this.cameraTranslateY - ((zAxis[1] ) * this.clipSpaceFactor * this.speed);
this.cameraTranslateZ = this.cameraTranslateZ + ((zAxis[2] ) * this.clipSpaceFactor * this.speed);
break;
}
case "s": { //strafe back
this.cameraTranslateX = this.cameraTranslateX + ((zAxis[0] ) * this.clipSpaceFactor * this.speed);
this.cameraTranslateY = this.cameraTranslateY + ((zAxis[1] ) * this.clipSpaceFactor * this.speed);
this.cameraTranslateZ = this.cameraTranslateZ - ((zAxis[2] ) * this.clipSpaceFactor * this.speed);
break;
}
case "a": {//strafe left
this.cameraTranslateX = this.cameraTranslateX - (xAxis[0] * this.clipSpaceFactor * this.speed);
this.cameraTranslateY = this.cameraTranslateY - (xAxis[1] * this.clipSpaceFactor * this.speed);
this.cameraTranslateZ = this.cameraTranslateZ + (xAxis[2] * this.clipSpaceFactor * this.speed);
break;
}
case "d": { //strafe right
this.cameraTranslateX = this.cameraTranslateX + (xAxis[0] * this.clipSpaceFactor * this.speed);
this.cameraTranslateY = this.cameraTranslateY + (xAxis[1] * this.clipSpaceFactor * this.speed);
this.cameraTranslateZ = this.cameraTranslateZ - (xAxis[2] * this.clipSpaceFactor * this.speed);
break;
}
case "q": { //strafe up
this.cameraTranslateX = this.cameraTranslateX + (yAxis[0] * this.clipSpaceFactor * this.speed);
this.cameraTranslateY = this.cameraTranslateY + (yAxis[1] * this.clipSpaceFactor * this.speed);
this.cameraTranslateZ = this.cameraTranslateZ - (yAxis[2] * this.clipSpaceFactor * this.speed);
break;
}
case "e": { //strafe down
this.cameraTranslateX = this.cameraTranslateX - (yAxis[0] * this.clipSpaceFactor * this.speed);
this.cameraTranslateY = this.cameraTranslateY - (yAxis[1] * this.clipSpaceFactor * this.speed);
this.cameraTranslateZ = this.cameraTranslateZ + (yAxis[2] * this.clipSpaceFactor * this.speed);
break;
}
case "z": { //roll left
this.cameraRotateZ = (this.cameraRotateZ + (this.sensitivity * this.speed)) % 360;
break;
}
case "c": { //roll right
this.cameraRotateZ = (this.cameraRotateZ - (this.sensitivity * this.speed)) % 360;
break;
}
}
I left the raymarching function intact. This gave me exactly what I wanted.