Lua Love: Mapping from isometric screen coordinates to game coordinates results in "off by one" x-coord

Source code used here: https://github.com/omrilevia/lua-iso

I've been playing around mapping screen coordinates to isometric coordinates and vice versa, following this tutorial: https://pikuma.com/blog/isometric-projection-in-games

I have some basic vec2 and mat2 classes which know how to do dot products, scaling, transforms, and inverses:

    Vec2 = Object:extend()
    
    function Vec2:new(x, y)
        self.x = x
        self.y = y
    end
    
    function Vec2:dot(vec)
        return self.x * vec.x + self.y * vec.y
    end
    
    function Vec2:scale(scalar)
        self.x = scalar * self.x
        self.y = scalar * self.y
        return self
    end

And the Mat2 class: (sorry for the terminology, I come from Java world)

Mat2 = Object:extend()

function Mat2:new(vec1, vec2)
    self.vec1 = vec1
    self.vec2 = vec2
end

function Mat2:transform(vec)
    local xPrime = self.vec1:dot(vec)
    local yPrime = self.vec2:dot(vec)

    return Vec2(xPrime, yPrime)
end

function Mat2:scale(scalar)
    self.vec1 = self.vec1:scale(scalar)
    self.vec2 = self.vec2:scale(scalar)
    return self
end

function Mat2:inverse()
    --[[d    -b]]
    --[[-c    a]] 
    return Mat2(Vec2(self.vec2.y, -self.vec1.y), Vec2(-self.vec2.x, self.vec1.x))
        :scale(1 / ((self.vec1.x * self.vec2.y) - (self.vec1.y * self.vec2.x)))
end

Here is the "Iso" class I use to transform from a game coordinate to a screen coord:

Iso = Mat2:extend()

function Iso:new(tileWidth, tileHeight)
    Iso.super:new(Vec2(1/2, -1/2):scale(tileWidth), Vec2(1/2, 1/2):scale(tileHeight))
end

function Iso:transform(vec)
    return Iso.super:transform(vec)
end

function Iso:scale(scalar)
    return Iso.super:scale(scalar)
end

function Iso:inverse()
    return Iso.super:inverse()
end

That's the math stuff. As far as drawing tiles on the screen, I have a "Grid" class which arranges tiles in a 2d table, and tells them their game coordinate like (1, 1) up to (10, 10).

The tiles themselves know how to draw themselves on the screen:

require "src.model.GameObject"
require "src.math2d.iso"
Tile = GameObject:extend()

function Tile:new(texId, pos)
    --texId 7 is broken
    if texId == 6 then
        texId = texId + 1
    end
    local constants = Constants()
    Tile.super.new(self, texId, pos)
    self.tilePath = constants.TILE_ASSET_PATH .. "tile-" .. texId .. ".png"
    print(self.isDirty)
end

function Tile:load()
    self.image = love.graphics.newImage(self.tilePath)
end

function Tile:update(dt)
end

function Tile:draw()
    local constants = Constants()
    -- transform the tile's position to isometric coords
    local zOffset = constants.MAX_TILE_HEIGHT - self.image:getHeight()

    local iso = Iso(constants.TILE_WIDTH, constants.TILE_HEIGHT)
    local vecIso = iso:transform(self.pos)
    print(iso:inverse():transform(vecIso).x)

    love.graphics.draw(self.image, constants.X_OFFSET + vecIso.x, constants.Y_OFFSET + vecIso.y + zOffset)
end

All of this works nicely according to the tutorial, and I'm able to get a nice looking tile grid:

I added a function to turn the given screen coordinate back into a game coordinate by taking the screen coordinate, subtracting off the x and y offset (not sure about the effect of the zOffset by the varying image heights), and transforming that vector by the inverse of the Iso matrix:

  function Screen:getGameCoordAt(pos)
    local constants = Constants()
    -- pos is isometric screen cord.
    -- perform inverse transform on pos, and reverse x and y offsets. 
    local iso = Iso(constants.TILE_WIDTH, constants.TILE_HEIGHT)
    return iso:inverse():transform(Vec2(pos.x - constants.X_OFFSET, pos.y - constants.Y_OFFSET))
end

Using this function, during the Grid update, I check to see if a current game coordinate is highlighted, and if so, I try to turn off that tile. In order to get the current tile that's highlighted, I take a math.floor() of transformed screen coordinates. The interesting thing is, I only get it to work nicely if I subtract 1 from the returned x game coordinate:

function Grid:update(dt)
    local constants = Constants()
    local mouse = Mouse()
    local screen = Screen()
    local gameCoord = screen:getGameCoordAt(mouse:getPos())
    local gridCoord = Vec2(math.floor(gameCoord.x) - 1, math.floor(gameCoord.y))
    if gridCoord.x >= 1 and gridCoord.x <= constants.GRID_SIZE then
        self.highlighted = self.grid[gridCoord.x][gridCoord.y]
    end
end

Example:

The origin seems to begin at (2, 1) rather (1, 1). I tried removing the code which adds in the height of the tiles, but it didn't seem to do the trick. Understandably this could be what's causing it since I'm only subtracting the xOffset and yOffset from the screen coord, but not the zOffset, since I don't know what the zOffset of the current tile is.

So my question is, why? Subtracting 1 from the transformed and floored screen coordinate beautifully matches the current tile's game coordinate but I'm having trouble wrapping my head around the cause.

Edit: I forgot to realize that images in love are drawn in such a way where their position is defined by the left corner of the image. I played around with constant image height tiles to remove the zOffset variability, and when I draw the tiles, I just smidge them to the left by tileWidth/2, so the origin of the game coordinates begins at the "tip" of the isometric diamond:

function Tile:draw()
    local constants = Constants()
    -- transform the tile's position to isometric coords
    local zOffset = constants.MAX_TILE_HEIGHT - self.image:getHeight()

    local iso = Iso(constants.TILE_WIDTH, constants.TILE_HEIGHT)
    local vecIso = iso:transform(self.pos)
    assert(iso:inverse():transform(vecIso).x == self.pos.x)
    assert(iso:inverse():transform(vecIso).y == self.pos.y)

    love.graphics.draw(self.image, constants.X_OFFSET + vecIso.x - constants.TILE_WIDTH/2, constants.Y_OFFSET + vecIso.y)
end

I also added in some asserts to verify the transform is in fact working. Now when hovering over the tip of an isometric tile, it will map neatly the tile's game coordinate:

Extending further, I was able to get it to work pretty well with images of different heights offsetting the screen coordinate's y position by the difference between the tile height and the max image height:

function Screen:getGameCoordAt(pos)
    local constants = Constants()
    -- pos is isometric screen cord.
    -- perform inverse transform on pos, and reverse x and y offsets. 
    local iso = Iso(constants.TILE_WIDTH, constants.TILE_HEIGHT)
    local transformed = iso:inverse():transform(Vec2(pos.x - constants.X_OFFSET, pos.y - constants.Y_OFFSET - 
        constants.MAX_TILE_HEIGHT + constants.TILE_HEIGHT))
    return Vec2(transformed.x, transformed.y)
end

It's not perfect, but it highlights tiles pretty well. Going forward, it looks like it makes the most sense to use images of fixed height.

I haven't yet wrapped my head around it all the way, but I think I have got it figured out. I do believe that this is the solution:

function Screen:getGameCoordAt(pos)
    local constants = Constants()
    -- pos is isometric screen cord.
    -- perform inverse transform on pos, and reverse x and y offsets. 
    local iso = Iso(constants.TILE_WIDTH, constants.TILE_HEIGHT)
    local transformed = iso:inverse():transform(Vec2(pos.x - constants.X_OFFSET, pos.y - constants.Y_OFFSET - 
        constants.MAX_TILE_HEIGHT + constants.TILE_HEIGHT))
    return Vec2(transformed.x, transformed.y)
end

Whenever the tiles are drawn, their game coords are transformed to iso, and they are given a base x and y offset. No problem if all the images and tiles are the same height. When we map them back, we just subtract off the offsets first.

But since there are tiles of different heights, they are given a zIndex, or zOffset. The higher the zOffset, the "lower" they are on the screen, since +y goes down:

function Tile:draw()
    local constants = Constants()
    -- transform the tile's position to isometric coords
    local zOffset = constants.MAX_TILE_HEIGHT - self.image:getHeight()

    local iso = Iso(constants.TILE_WIDTH, constants.TILE_HEIGHT)
    local vecIso = iso:transform(self.pos)
    assert(iso:inverse():transform(vecIso).x == self.pos.x)
    assert(iso:inverse():transform(vecIso).y == self.pos.y)

    -- scoot the image to the left by TILE_WIDTH/2, so that the center of the image lies on the origin. 
    love.graphics.draw(self.image, constants.X_OFFSET + vecIso.x - constants.TILE_WIDTH/2, constants.Y_OFFSET + vecIso.y + zOffset)
end

The zOffset is the difference between the base tile height, and the max tile height. Shorter tiles are pushed down by a maximum of MAX_TILE_HEIGHT - TILE_HEIGHT, and the tallest tiles have a zOffset of 0.

To get map the current screen coord to the correct base tile coord, we have to subtract the entirety of the zOffset and add back on the min tile height, and the cursor will align perfectly with the base of tile. This is great because the mapping only needs to know the minimum and maximum tile height.

Hopefully this makes sense, leave me a comment if I missed anything.