I'm trying to implement a checkpoint mechanism for my original script with the following structure:
import os
import sys
from dotenv import load_dotenv
def main():
load_dotenv()
checkpoint = int(os.getenv('CHECKPOINT'))
if checkpoint == 0:
...
do something
...
checkpoint += 1
os.environ['CHECKPOINT'] = str(checkpoint)
checkpoint = int(os.getenv('CHECKPOINT'))
if checkpoint >= 1:
...
do something
...
checkpoint += 1
os.environ['CHECKPOINT'] = str(checkpoint)
My .env file
CHECKPOINT=0
While this method is working inside a Windows machine (localhost), it is not working on my Ubuntu server.
That is, if my code fails somewhere on the checkpoint 2, I should be able to rerun it from that point only.
For example, if code fails at checkpoint=2:
Windows machine:
checkpoint=2
Ubuntu machine:
checkpoint=0
Also, I am using try and except to catch failures from the code, then exit system if there is any using:
sys.exit()
I can use a config.json to set these from time to time, but I really wanted to know why this is happening.
Here is a minimal working example:
import os
name = 'CHECKPOINT'
def getenv():
return int(os.getenv(name) or 0)
def putenv(value):
print(f'{name} (before): {getenv()}')
os.environ[name] = str(value)
print(f'{name} (after {value}): {getenv()}')
return getenv()
checkpoint = getenv()
if checkpoint == 0:
checkpoint = putenv(checkpoint + 1)
if checkpoint >= 1:
checkpoint = putenv(checkpoint + 1)
and when you call it:
$ CHECKPOINT=1 python3 1.py
CHECKPOINT (before): 1
CHECKPOINT (after setting 2): 2
$ echo -n $CHECKPOINT
$ python3 1.py
CHECKPOINT (before): 0
CHECKPOINT (after setting 1): 1
CHECKPOINT (before): 1
CHECKPOINT (after setting 2): 2
The CHECKPOINT environment variable is not persisted as it's stored in the first process (python3 1.py) which exits.