In bash, nvidia-smi command gives you information about the GPU.
We also have option to get this periodically such as nvidia-smi -lms 50
I want to get this info only as long as a particular process is running.
Pseudocode
nvidia-smi -lms 50 & > logfile.txt
(time ./process1) > timelog.txt
while process1 is running:
keep nvidia-smi running
kill nvidia-smi
How can I do this in bash, cleanly, such that once my bash script exits no process that starts here is left behind for me to clean?
A direct nvidia-smi based solution would be preferred to a bash based one, but the latter is also perfectly fine.
Run both in the background, then wait for the one your job depends on.
nvidia-smi -lms 50 > logfile.txt &
nvpid=$!
time ./process1 > timelog.txt &
prpid=$!
wait "$prpid"
kill "$nvpid"