windowscmdpathenvironment-variablespython-venv

Using a custom %Path% CMD vs venv's activate script


Win 10, Python 3.10.6

I planned on testing some Python-based AI tools on my main machine. However, to prevent interference with the main Python install, I've decided to set up a venv (using copies, NOT symlinks).

Once set up, venv instance needs to be activated. Documentation for Activate.bat script for venv states that it pushes a custom value to the Windows' %Path% env. variable.

I'd rather NOT have anything tinker with the %Path% on the main machine. So I've set up a simple "Launch venv-d Python.bat" script that runs a CMD with custom %Path%:

SET Path=<copy of the %Path%>;D:\venvEnv\Scripts;D:\venvEnv
start cmd

<copy of the %Path%> above is a 99% carbon copy of the system's Path, with Python directories omitted.D:\venvEnv is where the "virtual" install of the Python is.

Everything seems to work fine. Pip runs, and correctly displays clean list of modules (my main Python install has over 150 installed).

However, this is the workflow that I've learned from bash (on Linux). The questions remain:


Solution

  • The Windows Command Processor cmd.exe calls the Windows kernel library function CreateProcess with the function parameter lpEnvironment being a null pointer whenever an executable like python.exe is executed by cmd.exe. Each started executable gets therefore a copy of the environment variables currently defined in the memory of calling cmd.exe. The local PATH environment variable is copied for that reason to memory of every process created by this instance of cmd.exe.

    Please take a look on What is the reason for "X is not recognized as an internal or external command, operable program or batch file"? The batch file in the question is very reliable because of that is the standard behavior. The same happens on a batch file is started with a double click in Windows File Explorer, i.e. when explorer.exe calls CreateProcess to run %ComSpec% /c "Fully Qualified Batch File Name".

    I suggest further reading my answer on How to change directory with BAT file? It describes how to open a command prompt window with a shortcut file with setting up first a virtual Anaconda environment which does not require the modification of a persistent stored user or system environment variable used by lots of other processes like PATH.

    It is easy to find out what is modified by an activate batch of a virtual environment.
    There is executed in the command prompt window first:

    (echo CD="%CD%"& set) >"%USERPROFILE%\venv_before_activate.txt"
    

    Next is executed the activate batch file in same command prompt window. Then is run:

    (echo CD="%CD%"& set) >"%USERPROFILE%\venv_after_activate.txt"
    

    A text file comparison of the two generated text files %USERPROFILE%\venv_after_activate.txt and %USERPROFILE%\venv_before_activate.txt shows the differences applied on the execution environment of cmd.exe and all executables started next.

    1. Is the current working directory modified by the activate batch file as it can be seen on comparing the first line with CD= in the two text files?
    2. Are environment variables added like PYTHONPATH?
    3. Are environment variables modified like PATH?

    There can be next created a shortcut file to start the Windows Command Processor which sets up the required execution environment for the virtual environment and keep cmd.exe running as a command prompt for executing manually commands in this specific environment.

    Browse in Windows File Explorer to the directory %SystemRoot%\System32 which is usually C:\Windows\System32 and right click on the file cmd.exe. Next click with Windows 11 as operating system on the context menu item Show more options. Then click in context submenu Send to on the menu item Desktop (create shortcut). There is now on the user´s desktop the shortcut file cmd.exe - Shortcut. That file should be renamed by selecting it with a single left mouse button click and pressing key F2 to a meaningful name. Then press Alt+Enter or Alt+Return or right click on the renamed shortcut file and click in the context menu on Properties for opening the Properties dialog window of the shortcut.

    It is better using a shortcut file instead of a batch file to open a command prompt window with a specific defined local PATH. A shortcut file defines the parameters and values for the STARTUPINFO structure on explorer.exe calling CreateProcess. The shortcut property Target is passed with lpCommandLine, and the shortcut property Start in with lpCurrentDirectory to CreateProcess. The current working directory can be defined for that reason already with property Start in in the shortcut file. This directory path should be without surrounding " even on directory path containing a space or one of these characters &()[]{}^=;!'+,`~ as the string is passed directly via lpCurrentDirectory to CreateProcess which does not remove surrounding ".

    There can be a lot more configured with a shortcut file which is not possible with a batch file. The font, font size, number of rows and columns, foreground and background color and many other console properties can be defined in the properties of a shortcut file while the execution of a batch file with a double click results in using the console properties defined in the registry under the key HKEY_CURRENT_USER\Console. See the Microsoft developer blog Understanding Windows Console Host Settings for more details.

    The shortcut property Target could be in this case:

    %SystemRoot%\System32\cmd.exe /D /K if "%PATH:~-1%" == ";" (set "PATH=%PATH%D:\venvEnv\Scripts;D:\venvEnv") else set "PATH=%PATH%;D:\venvEnv\Scripts;D:\venvEnv"
    

    That command line starts the Windows Command Processor with following instructions:

    1. Ignore the registry values HKEY_CURRENT_USER\Software\Microsoft\Command Processor\AutoRun and HKEY_LOCAL_MACHINE\Software\Microsoft\Command Processor\AutoRun if being present in the registry because of option /D. Both AutoRun registry string values do not exist by Windows default and that is good so. There are unfortunately many installer scripts and advice which add automatically or suggest adding manually the AutoRun registry value for current user for running a batch file to setup a virtual environment. Those installation scripts and advice are bad because of the command line defined with the AutoRun registry value is executed always by cmd.exe independent if a user manually opens a command prompt window or a program/script executes cmd.exe for whatever reason in the background on not using the option /D as it is default on Windows. The AutoRun registry value modifies therefore the execution behavior of many scripts and applications running in background cmd.exe which often results in those scripts and applications are not working anymore as expected after adding the AutoRun registry value by an installer script or doing that manually after reading a bad advice to do so. The much better solution for opening a command prompt window and setup a specific execution environment is the usage of a shortcut file with a Target as defined above.
    2. Execute the IF condition which checks if the local environment variable PATH as defined by the Windows shell (explorer.exe) ends with a semicolon in which case the two folder paths D:\venvEnv\Scripts and D:\venvEnv are appended to local PATH of this instance of cmd.exe process without an additional semicolon. Otherwise, the two folder paths are appended to local PATH with the required semicolon after already predefined PATH string.
    3. Keep the command process running after execution of the IF condition and the appropriate command SET because of option /K for the executions of commands entered by the user in this command prompt window.

    The shortcut property Target could be defined also with:

    %SystemRoot%\System32\cmd.exe /D /K set "PATH=D:\venvEnv\Scripts;D:\venvEnv;%PATH%"
    

    That command line prepends the two folder paths to local PATH as defined by the Windows shell instead of appending them. That can make sense if many executables, libraries, or scripts used next often are in the two folders. But there must be checked if the two folders do not contain files with a file extension as listed in the environment variable PATHEXT which have the same name as a Windows command. In this case it is a bad idea to define a local environment variable PATH with first folder not being the most important folder %SystemRoot%\System32 containing the executables being a Windows command and the most often needed libraries of Windows.

    It would be also possible to define the shortcut property Target with:

    %SystemRoot%\System32\cmd.exe /D /K set "PATH=%SystemRoot%\System32;D:\venvEnv\Scripts;D:\venvEnv;%SystemRoot%;%SystemRoot%\System32\wbem;%SystemRoot%\System32\WindowsPowerShell\v1.0;%SystemRoot%\System32\OpenSSH"
    

    The local environment variable PATH is now defined completely specific for the command prompt window opened on using the shortcut with ignoring the environment variable PATH defined by the Windows shell with the concatenation of the system and the user environment variable PATH. The execution environment of the command prompt window opened on using the shortcut file becomes with that Target independent on future modifications of user and system environment variable PATH as stored in the Windows registry. That can be helpful on having multiple versions of Python installed in multiple directories.

    The Target property string can be extended also with more SET commands using the unconditional command operator & as described by single line with multiple commands in case of either environment variables must be defined or redefined in the local environment of started cmd.exe.

    It is advisable to also define the shortcut property Comment with a text explaining what this shortcut is for. The comment text is shown as tooltip on hovering the mouse pointer over the shortcut.