I have a server program (let's call it my-server
) I cannot alter and I am interested in measuring its maximum memory usage from when it starts to when a certain external event occurs.
I am able to do it manually on my shell by:
running /usr/bin/time -f "%M" my-server
triggering the external event
sending an INT
signal to the process by pressing CTRL
+ C
reading the standard error
I'd like to do this programmatically using Python. If not possible I'd like to find an alternative simple way to achieve the same result.
I tried launching the same command as a subprocess and sending an INT
signal to it:
import signal
import subprocess
proc = subprocess.Popen(
["/usr/bin/time", "-f", "%M", "my-server"],
stderr=subprocess.PIPE,
stdout=subprocess.PIPE,
text=True
)
# ...wait for external event
proc.send_signal(signal.SIGINT)
out, err = proc.communicate() # both out and err are ""
however both out
and err
are empty.
The same script works fine for processes that terminate:
proc = subprocess.Popen(
["/usr/bin/time", "-f", "%M", "echo", "hello"],
stderr=subprocess.PIPE,
stdout=subprocess.PIPE,
text=True
)
out, err = proc.communicate() # out is "hello", err is "1920"
but for processes that need to be terminated via signal I am not sure how to retrieve stderr after termination was issued. (or even before it, despite not useful)
The following somewhat equivalent example may be useful for testing:
terminate-me.py
import signal
import sys
def signal_handler(sig, frame):
print('Terminated', file=sys.stderr) # Need to read this
sys.exit(0)
signal.signal(signal.SIGINT, signal_handler)
print('Send SIGINT to terminate')
signal.pause()
main.py
import signal
import subprocess
proc = subprocess.Popen(
["python", "terminate-me.py"],
stderr=subprocess.PIPE,
stdout=subprocess.PIPE,
text=True
)
# No need to wait for any external event
proc.send_signal(signal.SIGINT)
out, err = proc.communicate()
print(out)
print(err)
It turns out I misunderstood /usr/bin/time
's behavior.
I thought that when receiving an INT
signal time
would relay it to its sub-process being measured, waited for it to terminate, and then terminated normally outputting its results.
What happens instead is that time
simply terminates before outputting any result.
When executing within a shell, pressing CTRL
+ C
actually sends an INT signal to both, but the sub-process terminates before time
does, so time
is able to complete its execution and it appears as if time
awaited the sub-process.
Therefore in my script I was able to make it work by only sending the INT signal to the sub-process: (actually, to all of its sub-processes for simplicity)
import signal
import subprocess
import psutil
proc = subprocess.Popen(
["/usr/bin/time", "-f", "%M", "my-server"],
stderr=subprocess.PIPE,
stdout=subprocess.PIPE,
text=True
)
# Wait for external event...
children = psutil.Process(proc.pid).children()
for child in children:
child.send_signal(signal.SIGINT)
out, err = proc.communicate()