Storing and Returning Lua Userdata

I have the following classes in C++

class B; 

class A {
    B* GetB();
    void SetB(B*& b) { _b = b;};
private:
    B* _b; 
}

And part of the lua binding code:

int A::setB(lua_State* L) {
    A* a = checkA(L,1) // Macro for luaL_checkudata
    B* b = checkB(L,2) // again similar macro
    a->SetB(b);        
    return 0;
}

int A::getB(lua_State* L) {
    A* a = checkA(L,1) // Macro for luaL_checkudata
    B* b = a->GetB();
    // how do i return the already created userdata for this B* instance?
    // right now I am doing
    B** bp = (B**)lua_newuserdata(L, sizeof(B*));
    *bp = b;       

    luaL_getmetatable(L, "B");
    lua_setmettable(L, -2);

    return 1; 
}

And I want to wrap these as userdata in Lua so I could do something like:

local a = a.new()    -- create new userdata
local b = b.new()    -- create new userdata
a:SetB(b)            -- associate B with A
local b2 = a:GetB()  -- get the B associated with A back, and stored as b2

When I print the addresses of b and b2 I get two unique address, which makes sense because I have called lua_newuserdata. But ideally I would want it to return the same userdata because they point to the same memory block. How would one do this?

I want Lua to be in 'charge' of the memory, so it will get properly deleted on garbage collection. So I don't know if light userdata is possible.

You can mirror the member variables of your C++ objects in Lua using a uservalue table. This way your methods become:

int A::setB(lua_State* L) {
    A* a = checkA(L,1) // Macro for luaL_checkudata
    B* b = checkB(L,2) // again similar macro
    a->SetB(b);
    // if you are paranoid about consistency, you should preallocate the
    // the uservalue table slot for "_b", so that the following code
    // can't fail during memory allocation
    lua_getuservalue(L, 1); // use lua_getfenv in Lua 5.1
    lua_pushvalue(L, 2); // duplicate B userdata on stack top
    lua_setfield(L, -2, "_b"); // store it in the uservalue table       
    return 0;
}

int A::getB(lua_State* L) {
    A* a = checkA(L,1) // Macro for luaL_checkudata
    lua_getuservalue(L, 1);
    lua_getfield(L, -1, "_b");
    return 1; 
}

This has the added advantage that the B userdata is not collected (causing a dangling pointer) while it is referenced by the A object.

Make sure that all A objects actually have a uservalue table by using lua_setuservalue() (lua_setfenv() for Lua 5.1) after each lua_newuserdata() for A objects.

You can also add checks to make sure that the C++ side and Lua side are still in sync (in case you keep using your interface from C++ as well), but if you find a B object that is not mirrored in the uservalue table, there is not much you can do except raise an error, because you don't know how it got there and who owns it.

Edit:

If you want to manage a container of B pointers, you can do so by storing all userdata in a table (the uservalue table itself, or a field in it) and mirroring all container operations, but this can get very error prone. As an alternative you can use the uservalue table as a mapping from C++ pointer value (lightuserdata) to Lua object (full userdata). This mapping will keep the B userdatas alive, and you can simply look them up by pointer value. You just have to update the mapping whenever you add or remove B objects. E.g.

int A::pushB(lua_State* L) {
    A* a = checkA(L,1)
    B* b = checkB(L,2)
    lua_getuservalue(L, 1);
    lua_pushlightuserdata(L, (void*)b);
    lua_pushvalue(L, 2);
    lua_rawset(L, -3);
    a->PushB(b);
    return 0; 
}

int A::popB(lua_State* L) {
    A* a = checkA(L,1)
    B* b = a->PopB();
    lua_getuservalue(L, 1);
    lua_pushlightuserdata(L, (void*)b);
    lua_rawget(L, -2); // push full userdata
    if (!a->ContainsB(b)) {
        // remove reference only if this b isn't in the container anymore
        // if done wrong b may be collected too soon (while A still has
        // a pointer stored), or too late (when object a gets collected) 
        lua_pushlightuserdata(L, (void*)b);
        lua_pushnil(L);
        lua_rawset(L, -4);
    }
    return 1;
}