When making Lua bindings for C++ classes, should I return tables or userdata objects?
Does anyone know any of the pros and cons for each method?
I recommend returning userdata. Regardless of approach, there has to be somewhere to put the pointer to the C++ data, or the actual C++ data itself, and there's nowhere safe to do this with a table.
Returning tables would make sense in some situations, because they can be 'annotated' in Lua with extra attributes without one's having to do anything extra to support this. Unfortunately the C++ object pointer has to go somewhere, and there's nowhere sensible for it to go other than an actual entry in the table itself.
This is not a very safe place for it to go. It can be found by Lua code, and removed or replaced. This could be by accident, or on purpose, it doesn't really matter.
My preference therefore is to return userdata objects. They can be made to work like tables if one really must insist upon that, but they also have a "secret" area (the actual userdata itself) where the C++ object pointer can be stored, safe from overwriting by Lua code.
(Userdata objects also have an "environment" pointer, which is another place to store object-specific data. As with the userdata payload itself, this value is inaccessible to Lua code and can't be damaged that way.)