unity-game-enginetransformhlslcggeometry-surface

Vertex position relative to normal


In a surface shader, given the world's up axis (and the others too), a world space position and a normal in world space, how can we rotate the worldspace position into the space of the normal?

That is, given a up vector and a non-orthogonal target-up vector, how can we transform the position by rotating its up vector?

I need this so I can get the vertex position only affected by the object's rotation matrix, which I don't have access to.

Here's a graphical visualization of what I want to do:

The diagram is bidimensional, but I need to solve this for a 3D space.

vector diagram


Solution

  • Looks like you're trying to rotate pos by the same rotation that would transform up to new_up.

    Using the rotation matrix found here, we can rotate pos using the following code. This will work either in the surface function or a supplementary vertex function, depending on your application:

    // Our 3 vectors
    float3 pos;
    float3 new_up;
    float3 up = float3(0,1,0);
    
    // Build the rotation matrix using notation from the link above
    float3 v = cross(up, new_up);
    float s = length(v);  // Sine of the angle
    float c = dot(up, new_up); // Cosine of the angle
    float3x3 VX = float3x3(
        0, -1 * v.z, v.y,
        v.z, 0, -1 * v.x,
        -1 * v.y, v.x, 0
    ); // This is the skew-symmetric cross-product matrix of v
    float3x3 I = float3x3(
        1, 0, 0,
        0, 1, 0,
        0, 0, 1
    ); // The identity matrix
    float3x3 R = I + VX + mul(VX, VX) * (1 - c)/pow(s,2) // The rotation matrix! YAY!
    
    // Finally we rotate
    float3 new_pos = mul(R, pos);
    

    This is assuming that new_up is normalized.

    If the "target up normal" is a constant, the calculation of R could (and should) only happen once per frame. I'd recommend doing it on the CPU side and passing it into the shader as a variable. Calculating it for every vertex/fragment is costly, consider what it is you actually need.

    If your pos is a vector-4, just do the above with the first three elements, the fourth element can remain unchanged (it doesn't really mean anything in this context anyway).

    I'm away from a machine where I can run shader code, so if I made any syntactical mistakes in the above, please forgive me.