What is the difference between eval $(opam env --switch={switch} --set-switch) and opam switch set SWITCH?

It seems we have two different ways to activate the current opam switch environment. So my questions are:

Which one is the preferred method?
What is the difference between the two? eval $(opam env --switch=$SWITCH --set-switch) vs opam switch set $SWITCH

Thanks!

Context

I need to change opam envs within python due to my applicaiton (no way around this 100%).

Usually I do:

eval $(opam env --switch={switch} --set-switch)

but this gives an issue (see end).

Thus, going to try:

opam switch set {switch}

are these truly equivalent?

(Note: in python opam switch set {switch} seems to work, but still like to understand why there are two version)

For context error:

Traceback (most recent call last):
  File "/lfs/ampere4/0/brando9/iit-term-synthesis/iit-term-synthesis-src/data_pkg/data_gen.py", line 510, in <module>
    main()
  File "/lfs/ampere4/0/brando9/iit-term-synthesis/iit-term-synthesis-src/data_pkg/data_gen.py", line 497, in main
    asyncio.run(create_dataset(path_2_save_new_dataset_all_splits=args.path_to_save_new_dataset,
  File "/dfs/scratch0/brando9/anaconda/envs/iit_synthesis/lib/python3.9/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/dfs/scratch0/brando9/anaconda/envs/iit_synthesis/lib/python3.9/asyncio/base_events.py", line 647, in run_until_complete
    return future.result()
  File "/lfs/ampere4/0/brando9/iit-term-synthesis/iit-term-synthesis-src/data_pkg/data_gen.py", line 437, in create_dataset
    coq_proj_data: DataCoqProj = await get_coq_proj_data(coq_proj, split)
  File "/lfs/ampere4/0/brando9/iit-term-synthesis/iit-term-synthesis-src/data_pkg/data_gen.py", line 194, in get_coq_proj_data
    path2filenames_raw: list[str] = strace_build_coq_project_and_get_filenames(coq_proj)
  File "/afs/cs.stanford.edu/u/brando9/pycoq/pycoq/opam.py", line 706, in strace_build_coq_project_and_get_filenames
    activate_opam_switch(switch)
  File "/afs/cs.stanford.edu/u/brando9/pycoq/pycoq/opam.py", line 892, in activate_opam_switch
    raise e
  File "/afs/cs.stanford.edu/u/brando9/pycoq/pycoq/opam.py", line 886, in activate_opam_switch
    res = subprocess.run(command.split(), check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
  File "/dfs/scratch0/brando9/anaconda/envs/iit_synthesis/lib/python3.9/subprocess.py", line 505, in run
    with Popen(*popenargs, **kwargs) as process:
  File "/dfs/scratch0/brando9/anaconda/envs/iit_synthesis/lib/python3.9/subprocess.py", line 951, in __init__
    self._execute_child(args, executable, preexec_fn, close_fds,
  File "/dfs/scratch0/brando9/anaconda/envs/iit_synthesis/lib/python3.9/subprocess.py", line 1821, in _execute_child
    raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'eval'

I think it has something to do with calling subprocesses from within python don't fully understand,

Why is subprocess not inheriting the env vars from the main python process but from the mutable subprocesses as more subprocess calls are done?

[quote="Frederic_Loyer, post:22, topic:10957"] A process can’t change the environment of an other process. Then opam can’t change the parent process environment (bash or Python). [/quote]

I've confirmed this. What I do is run opam switch set coq-8.10 from a python subprocess:

        #     opam_set_switch_via_opam_switch('coq-8.10')
        result = subprocess.run(f'opam switch set {switch}'.split(), check=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

which the docs says returns a completed process.

Then I compare the contents of the env variables of the subprocess by calling via another subprocess the cmd opam env and compare it with the main python process by comparing it with os.environ. I get that the two indeed don't match:

    # opam_env_dict: dict = get_variables_from_opam_env_output(py_prints_on=py_prints_on)
            result = subprocess.run('opam env'.split(), capture_output=True, text=True)
# ... compare with os.environ
            assert uutils.check_dict1_is_in_dict2(opam_env_dict, os.environ, verbose=True)

assert fails

--> k='OPAM_SWITCH_PREFIX' is in dict2 but with different value 
dict1[k]='/Users/brandomiranda/.opam/coq-8.10' 
dict2[k]='/Users/brandomiranda/.opam/test'

The only thing that confuses me is that it seems that subprocess has it's own process that does remember things. I say this because I would have expected the new subprocess that calls opam env to not be affected by the first opam switch set coq-8.10 but it seems it was affected. I expected the 2nd subprocess to spawned from the main python and be independent form the process that called opam switch set coq-8.10.

refs:

Solution

You seem to use a Python function which tries to find an eval program. Such a program doesn't exist. It is an internal command of a Bourne Shell. This explains the error.

Given the way opam works, what should be useful to do is:

Launch opam env --switch my_switch --set-switch,
Analyse its output (multiple lines with a VAR=value syntax, some ; separators and export command to be ignored)
Change the environment (os.environ[var] = value)

Afterward, the right ocaml will be found in the modified PATH and will be setup properly.

Back to the original question : What is the difference between eval $(opam env --switch={switch} --set-switch) and opam switch set SWITCH?

There are 3 ways to set the current opam switch : using a --switch option with all your opam commands, setting the OPAMSWITCH environment variable ($(opam env --switch={switch} --set-switch) does this), and setting a global state stored in your .opam directory with opam switch set.

The three ways work well with the opam command, but Ocaml programs (ocamlc, opcamlopt, ...) are not selected automatically when you set a switch. Then you have two ways to make your selected switch efficient : Using opam exec with every command, or using eval $(opam env) or eval $(opam env --switch SWITCH --set-switch) to set the PATH variable and some others. If you have an interactive shell, and some Ocaml hooks, eval $(opam env) will be implicit, then a opam switch set will be the most handy way of setting an other switch.

The eval command MUST be executed in a shell father of all the commands you want to execute in the selected switch. Then from a Python program, either execute opam exec with all commands, or mimick eval like proposed in the first part of the answer.