I am trying to get a direction vector to where my cursor is on the screen, however it gives me large values instead of the actual values. When doing mouse picking I am getting extremely small numbers for my world coordinates such as
Mouse is pointing at World X: 4.03225e-05 Y: -0.00048387 Z: -1
Am I doing something wrong here, I have adapted this code from here. I am using glm::intersectLineTriangle()
to test if the line is pointing at the triangles world coordinates in question.
Here is the relevant code:
double mouse_x, mouse_y;
glfwGetCursorPos(GL::g_window, &mouse_x, &mouse_y);
int width, height;
glfwGetFramebufferSize(GL::g_window, &width, &height);
// these positions must be in range [-1, 1] (!!!), not [0, width] and [0, height]
float mouseX = (float)mouse_x / ((float)width * 0.5f) - 1.0f;
float mouseY = (float)mouse_y / ((float)height * 0.5f) - 1.0f;
glm::mat4 invVP = glm::inverse(projection * view);
glm::vec4 screenPos = glm::vec4(mouseX, -mouseY, 1.0f, 1.0f);
glm::vec4 worldPos = invVP * screenPos;
glm::vec3 mouseClickVec = glm::normalize(glm::vec3(worldPos));
std::cout << "Mouse is pointing at World X: " << mouseClickVec.x << " Y: " << mouseClickVec.y << " Z: " << mouseClickVec.z << '\n';
glm::vec3 intersect;
if(glm::intersectLineTriangle(glm::vec3(0.0f, 0.0f, 0.0f), mouseClickVec, m_vertices_world[0], m_vertices_world[0], m_vertices_world[0], intersect))
{
std::cout << "Intersect at X: " << intersect.x << " Y: " << intersect.y << " Z: " << intersect.z << '\n';
setColor(glm::vec4(0.0f, 0.0f, 1.0f, 1.0f));
}
else
{
setColor(glm::vec4(1.0f, 0.0f, 0.0f, 1.0f));
}
When using orthographic (parallel) projection, the point of view is not (0, 0, 0), (not in view space and of course not in world space). You have to create a ray from the near plane (-1) to the far plane (1):
glm::vec4 worldPosFar = invVP * glm::vec4(mouseX, -mouseY, 1.0f, 1.0f);
glm::vec3 mouseClickVecFar = glm::normalize(glm::vec3(worldPosFar));
glm::vec4 worldPosNear = invVP * glm::vec4(mouseX, -mouseY, -1.0f, 1.0f);
glm::vec3 mouseClickVecNear = glm::normalize(glm::vec3(worldPosNear));
if (glm::intersectLineTriangle(mouseClickVecNear, mouseClickVecFar,
m_vertices_world[0], m_vertices_world[0], m_vertices_world[0], intersect))
{
// [...]
}