I have a question about where the output of a certain command is going. I'm using restic
as an example of a command that behaves this way. It's the last line of the command that's of interest to me, because it updates while the restore process is working.
For context, restic's a back-up utility; the command below is restoring the contents of my-repo
from a backup stored on example.com
back to my local machine.
$ restic restore latest --repo sftp:matt@example.com:my-repo --target . --verbose
repository ae412afe opened (version 1)
restoring <Snapshot dff2f51a> to /Users/matt
[0:11] 0.02% 192 files 37.687 MiB, total 39644 files 171.389 GiB
The last line updates over time until it reaches 100%, e.g.
[12:31] 1.22% 2989 files 2.087 GiB, total 39644 files 171.389 GiB
I want to run the restore command from a Python script (using subprocess.Popen
) and want the script to print out the updating line but haven't been able to get that working.
I suspected it could be that the updating line was being output to stderr instead of stdout. So I repeated the restore command and redirected the stdout and stderr to files, as below:
restic restore lat...rbose > out.log 2> err.log
err.log
remains empty, and out.log
only contains the restoring <Snapshot dff2f51a> to /Users/matt
line, nothing else.
Here's the snippet of my code calling Popen:
backup_command = [
# fmt: off
"restic",
"--repo", repo_name,
"backup",
"--verbose",
"--exclude", "node_modules",
"--exclude", "whatever",
# fmt: on
]
backup_process = subprocess.Popen(backup_command, env=env_vars)
backup_process.wait()
In reference to jurez's answer below, I'm not passing stdin
or stderr
, but having looked at the source for Popen, they default to None
, as per jurez's suggestion.
I have a couple of questions.
repository ae412afe opened (version 1)
line in either of my output files?[0:11] 0.02% 192 fil...
line going? And how can I show it?From the restic documentation:
When running from a non-interactive console progress reporting is disabled by default to not fill your logs. For interactive and non-interactive consoles the environment variable
RESTIC_PROGRESS_FPS
can be used to control the frequency of progress reporting. Use for example 0.016666 to only update the progress once per minute.
My use of Popen
creates a non-interactive shell, resulting in no progress reporting output. Setting the value works as advertised. I find it easier to think of as "seconds between each update" rathen that "frames per second" so use 1 / INTERVAL_IN_SECONDS
to set the FPS.
Credit to jasonharper for looking up the docs, thanks!