I need a better method or I need to learn how to use the method
I'm using more correctly. Normally, when I press cntrl-c
my work gets pickled, but one time it did not. The subsequent time that I tried to open the pickle I got a ran out of input
error. I need to know why this happened so that it does not happen again. As I'm running my code, every one hundred loops I will save the pickle and if I hit KeyboardInterrupt, theoretically my work is supposed to be pickled before the program stops. My hunch is that if I press KeyboardInterrupt
while pickle.dump(obj, temp)
is being executed then the file will overwrite the old file but as it's overwriting if the program is killed in the middle of the overwrite then the file will be sort of half-written. What I also do not understand is that after I hit KeyboardInterrupt
the program should execute the line print("help me")
but it's not, at least on the sole occasion where I tried that.
import pickle
def save_pickle(obj, file):
temp = open(file, "wb")
##i have a feeling that if i hit keyboard interrupt
##while the below line is being executed that it won't
## save the pickle properly
pickle.dump(obj, temp)
temp.close()
class help_me():
def __init__(self):
pass
def func1(self):
try:
self.func2()
except KeyboardInterrupt:
pass
print('help me') # this line did not get activated
save_pickle(obj, file)
def func2(self):
#this function will take several days
for i in range(600_000):
time.sleep(1)
if i % 100 == 0:
save_pickle(obj, file)
As noted in the comments, if you terminate your program with SIGINT (i.e. CTRL+C) mid-way through pickling an object to a file, you may end up with a partially written (corrupt) file.
To avoid this, you can customize how your program responds to SIGINT so that the critical parts don't stop abruptly.
Before the loop, register a temporary signal handler for SIGINT that sets a "stop" flag instead of raising KeyboardInterrupt
. In the loop, check for the stop flag, and break when it's set. After the loop, you can reset the signal handler to whatever it was before the loop.
def func2(self):
stop = False
def handle_sigint(_sig, _frame)
nonlocal stop
print("setting stop flag")
stop = True
original_handler = signal.getsignal(signal.SIGINT)
signal.signal(signal.SIGINT, handle_sigint)
while not stop:
time.sleep(1)
save_pickle(obj, file)
signal.signal(signal.SIGINT, original_handler)