pythonenvironment-variablespython-os

How to differentiate and split `os.environ` into defaults and addons


Is it possible to split os.environ into default environment variables and custom addon variables?

For example, using syntax of sets representing the key/ENV_VAR_NAME:

custom_addon_env_var_keys = set(os.environ.keys()) - set(os.environ.default.keys()) # is there this `os.environ.default` kinda thing?

Is there some kind of os.environ.default to get pure environment variables, after the program has started (that is, exclude all the custom keys I've added since the program started)?

I've tried looking into the source code of os module, the environ is initialized by a function _createenviron, but it's been deleted as soon as the environ been initialized, so it's impossible to re-initialize a environ_2 and do the job, and copying the whole initializing function looks like a dumb way.

The following code gives a brief-view of the expecting behavior. I'm using this for cross file variables, so saving a original copy at start is not what I need:

import os

# I'm using this for cross file variables
# so it is impossible to save a original copy at start
ORIGINAL_ENV_COPY = os.environ.copy()

os.environ["MY_CUSTOM_ENV_KEY"] = "HelloWorld"

# the "ORIGINAL_ENV_COPY" should be a method of getting the original version of "os.environ"
custom_addon_env_var_keys = set(os.environ.keys()) - set(ORIGINAL_ENV_COPY.keys())

print(custom_addon_env_var_keys) # output: {"MY_CUSTOM_ENV_KEY"}

Edit

My apologies not giving enough information in the post:

I have a file named environment.py that will initialize constants, and it will be packed into binary executable along with main.py, here's a part of it:

import os

os.environ["TIMEZONE"] = "Asia/Taipei"

os.environ["PROJECT_NAME"] = "Project name"

os.environ["APP_USER_MODEL_ID"] = f"{os.environ['PROJECT_NAME']}.App.User.Model.Id"

os.environ["SERVER_URL"] = f"https://www.my.server.io/projects/{os.environ['PROJECT_NAME']}"
os.environ["STORAGE_URL"] = f"https://www.my.server.io/projects/{os.environ['PROJECT_NAME']}/Storage"
os.environ["MODULES_URL"] = f"https://www.my.server.io/projects/{os.environ['PROJECT_NAME']}/Modules"

os.environ["USER_AGENT"] = random.choice([
    "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/68.0.3440.106 Safari/537.36",
    "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/67.0.3396.99 Safari/537.36",
    "Mozilla/5.0 (Windows NT 10.0; WOW64) Gecko/20100101 Firefox/61.0",
    "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36",
    "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.62 Safari/537.36",
    "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.101 Safari/537.36",
    "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)",
    "Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.2.15) Gecko/20110303 Firefox/3.6.15"
])

if getattr(sys, "frozen", False):
    sys.argv = [arg for arg in sys.argv if not arg.startswith("-")]
    os.environ["EXECUTABLE_ROOT"] = os.path.dirname(sys.executable)
else:
    os.environ["EXECUTABLE_ROOT"] = os.path.dirname(os.path.abspath(sys.modules["__main__"].__file__))

Why do I need to store variables and constants into os.environ?

  1. This environment.py will be packed into binary executable along with main.py, and therefore, to all other modules, they don't even know there's even a file called environment.py (since now, all that's left is main.exe)
  2. As you can see, some of the variables, e.g. USER_AGENT, EXECUTABLE_ROOT, takes quite some space, if don't store it, there will be alots of duplicated codes.

Finally why do I need to get custom add on env keys

Because I store variables in it, when receiving crash reports, I need debug if the problem is caused by any of them.

Take the USER_AGENT as an example, if one of them is no longer available, I have to know which one is chosen and raised the error, and then remove it from the list, instead of testing them one by one.

And there are a lot of default environment variable keys, which if I send everything in os.environ would be a mess, I only need the variable keys I've added.

Answer (thanks to @zondo)

Save the original os.environ under os using setattr, and in order to prevent re-import overwriting the original save (mentioned by the comment from @wim), I've put a if statement to judge it.

import os

# if this is the first time running
# there won't be a "default_environ" attr
if not hasattr(os, "default_environ"):
    setattr(os, "default_environ", os.environ.copy())

# ... other variables

Solution

  • Python modules are only ever imported once within the same program instance. It doesn't matter how many different files you have importing something, that first time it's imported is the only time it's run, and everything else reuses it. There are two ways to take advantage of this.

    First, the more clean way, which is choose a file to set everything you want. For example, startup.py:

    import os
    
    environ = os.environ.copy()
    # other settings you want
    

    And then all your other files can use:

    import os
    import startup
    
    new_vars = set(os.environ) - set(startup.environ)
    

    The second way is really more of a hack, and I don't like it, but if the environ is the only thing you're doing, it would be easier. And that's just this:

    import os
    
    os.startup_environ = os.environ.copy()
    

    Again, since the os import will be the same across everything, that variable will be accessible everywhere.