Lua "attempt to index a nil value": indexing nested tables, where several tables might be not existing (using __index and __newindex metamethod)

I have a lua function called from c, giving it a table as argument with some string fields and number values. I serialized some world state from my c program in this table. This is for a dialogue system. When I access the table in lua, i want to check whether or not some world state matches. This controls the output of the dialogue options (see example code below).

What I don´t want is to pre initialize all possible state.

I only want to ask in lua for example:

if(table.player.killedEnemys > 5) then
    //calls a c-function that loads the given Text to display and the follow up function when clicked
    PresentOption("Try to impress opponent.", "show_off_to_soldier")
end

It works fine. If a player field was setted in the table before calling the function with this table as argument, killedEnemys is 0 due to the metamethod __index and __newindex:

function newindex_metamethod(t,i, v)
    rawset(t, i, v)
end

function index_metamethod(t,i)
    t[i] = 0
end

But I encounter an error by the line of the if-condition, saying: attempt to index a nil value (field 'player') when player is also not existing in the table so far.

I thought the metamethods set the field "player" in the table before indexing this field further with "killedEnemys". But this seems to be wrong and I do not understand why.

In my imagination the table should looks like this:

table =
        {
            "player" =                          //set by the metamethods
                       {
                            "killedEnemys" = 0  //set by the metamethod
                       }
        }

Questions: 1. Is it possible to use metamethods in lua like this, if more than one field in a nested table does not exist? 2. How does Lua operate on this nested table calls?

This is how I call it from C and the function being called in lua:

//IN C:
        lua_getglobal(L, funcName);           // where funcName represents a string for the function
                                              //"small_talk" below
      
        lua_table_serializer ser = {};        //serialization of the game state using the LBP-Method
        ser.L = L;                            //it creates a table and pushes it to the stack where
        Serialize(&ser, npc);                 

        lua_newtable(L);                 // creating a table and set it as metatable of the
        lua_pushstring(L, "__index");    // table with the serialized game state
        lua_getglobal(L, "index_metamethod");
        lua_rawset(L, -3);
        lua_pushstring(L, "__newindex");
        lua_getglobal(L, "newindex_metamethod");
        lua_rawset(L, -3);
        lua_setmetatable(L, -2);

        lua_pcall(L, 1, 0, 0);

//IN LUA:
function small_talk(table)
    PlayText("....")
    PresentOption("...", "small_talk2")
    if(table.player.killedEnemys > 5) then
        PresentOption("Try to impress opponent.", "show_off_to_soldier")
    end
end

There are no other global variables with the same name.

I also tried some variations for the metamethods like setting t[i] = {} and so on.

The metamethods are called for field player, but not killedEnemys (I printed the metamethod calls to check).

Anything works fine unless there are multiple levels of not existing fields.

I also saw this question which is similar. But with the solutions the syntacic sugar is lost. Also I want to know why the metamethod-solution does not the job.

There are a few problems here that build on each other.

For starters, when an __index metamethod is invoked, and that metamethod is a function, the value resolved by that index is the return value of the function.

For example,

local t = setmetatable({}, {
    __index = function (self, key)
        return 42
    end
})

print(t.a, t.b)

prints

42  42

So in your code, even if you assign a value to that key in the table

function index_metamethod(t,i)
    t[i] = 0
end

the metamethod returns nothing, and the result of table.player is nil. You need to return t[i] (or equivalent) after you set it.

---

Next, nested indexing does not repeatedly invoke the metamethod(s) of the root table, but rather it invokes the metamethod(s) of each individual table in the chain of access.

That is, for a.b.c, if 'b' does not exist in a, but a has an __index metamethod, then a.b is resolved to the result of invoking that metamethod (as previously discussed). Then, that intermediate value (a.b) is indexed by 'c', and again if 'c' does not exist in that intermediate value, then that intermediate value's __index metamethod would be invoked, assuming it exists.

Basically a.b.c is

local result
do
    local x = a.b -- possibly invoke `getmetatable(a).__index`
    result = x.c  -- possibly invoke `getmetatable(x).__index`
end

Any table you want keys to be auto-magically generated in must have its own metatable and metamethods set that do just that (though the metatable can possibly be reused by more than one table).

This issue is very similar in concept to this question from just a few days ago, which shows how to solve this on the Lua side.

---

Finally, assuming you fix everything else, the value set by index_metamethod (t[i] = 0) means table.player will resolve as 0, and so

table.player.killedEnemys

will create the error

attempt to index a number value (field 'player')

but "fixing" (assuming you use the same index_metamethod for every nested table) this by changing t[i] = 0 to t[i] = {} means that

table.player.killedEnemys > 5

will create the error

attempt to compare number with table

It would be possible to address this by implementing more metamethods (e.g., __call, __lt, __eq, etc.), but you still need a way to decide what each key's true value is at any given point in time.

Basically, at some point you will need a way to distinguish keys from one another and decide what their actual value should be, and how to access it. This is an issue of design, and thus very subjective, so it is really on you to figure out how to accomplish this.