How can I load a file of lua tables and variables without polluting the global environment? Since doing a loadfile and running it just loads everything in the global space and may overwrite something else which I don't want.
In Lua 5.1 and without much error handling you could do this:
-- load and run a script in the provided environment
-- returns the modified environment table
function run(scriptfile)
local env = setmetatable({}, {__index=_G})
assert(pcall(setfenv(assert(loadfile(scriptfile)), env)))
setmetatable(env, nil)
return env
end
The first line creates an empty environment table that can see all existing globals, but which cannot trivially change them since they are visible only by proxy through the __index metamethod. Any globals the script creates would be stored in env, which is returned. This will work well for simple scripts that just set a bunch of configuration parameters, and which might need to call simple safe functions to set them based on conditions at run time.
Note that making the globals visible to the script is a convenience. Although the globals cannot be modified from the script in the obvious way, _G is a global variable that contains a reference to the global environment (containing _G._G, _G._G._G, etc...) and _G can be modified from the script which could lead to further issues.
So rather than using _G for the index, it would be much better to construct a table that contains only functions known to be safe and known to be needed by your script's author.
A complete solution would be to run the script in a sandbox, and possibly further protected to prevent accidental (or deliberate) denial of service or worse. Sandboxes are covered in more detail at the Lua User's Wiki. The topic is deeper than it seems at first glance, but as long as your users are trusted to be non-malicious then practical solutions are straightforward.
Lua 5.2 changes things a little bit by eliminating setfenv() in favor of a new parameter to load(). Details are also in the wiki page.