pythonmultithreadingimportshared-variable

How to import a shared variable in two separate scripts?


I have the following simplified python code:

myproject
├── main.py
├── shared_var.py
└── utils.py

shared_var.py:

FOO = 'INIT'

utils.py

import time
from shared_var import FOO

def func_b():
    global FOO
    print('func_b initial FOO value: ', FOO)
    for i in range(10):
        FOO = str(i)
        print('func_b FOO value: ', FOO)
        time.sleep(1)

    FOO = 'DONE'
    print('func_b FOO value: ', FOO)

main.py:

import time
from shared_var import FOO
from utils import func_b
import threading


def check_FOO_status():
    print('main current FOO: ', FOO)
    if FOO == 'DONE':
        return True
    else:
        return False

if __name__ == "__main__":
    print('main start FOO value: ', FOO)

    t = threading.Thread(target=func_b)
    t.start()

    running = True
    while running:
        if check_FOO_status():
            break
        time.sleep(3)

    print('main end FOO value: ', FOO)

This is a oversimplified version of the problem I was trying to solve in my application. Basically what I am trying to do is:

  1. start a child thread in main.py, and the child thread will perform a time-consuming task asynchronously.
  2. the child thread updates a global variable FOO to keep track of progress while processing the task.
  3. The main is not blocked by the time-consuming task, after it spawns a child thread to do this task, it keeps on to complete some other tasks. And then it keeps checking the global variable FOO to see the progress of the child thread.

However, the output is as follows:

/usr/local/bin/python3.6 /Users/john/myproject/main.py
main start FOO value:  INIT
func_b initial FOO value:  INIT
func_b FOO value:  0
main current FOO:  INIT
func_b FOO value:  1
func_b FOO value:  2
main current FOO:  INIT
func_b FOO value:  3
func_b FOO value:  4
func_b FOO value:  5
main current FOO:  INIT
func_b FOO value:  6
func_b FOO value:  7
func_b FOO value:  8
main current FOO:  INIT
func_b FOO value:  9
func_b FOO value:  DONE
main current FOO:  INIT
main current FOO:  INIT
main current FOO:  INIT
main current FOO:  INIT
main current FOO:  INIT
main current FOO:  INIT
main current FOO:  INIT
main current FOO:  INIT
main current FOO:  INIT
main current FOO:  INIT
...(infinite loop)

As you can see, obviously there are two copies of the global variable FOO. The main and the child thread is not sharing the single global variable. I think it is an issue with importing but after some search online I couldn't find a cause.

Any suggestions?


Solution

  • Instead of writing from shared_var import FOO just write import shared_var, and when changing/reading this FOO variable use shared_var.FOO instead of just FOO. In this case we use the fact that if in some module was already import previously, and then we try to import it again, python won't import it again, but it will just give you reference to already existing object in memory. Looks like this doesn't work for import from construction, so you need to use import.

    Here's the working code:

    main.py

    import time
    import shared_var
    from utils import func_b
    import threading
    
    
    def check_FOO_status():
        print('main current FOO: ', shared_var.FOO)
        if shared_var.FOO == 'DONE':
            return True
        else:
            return False
    
    if __name__ == "__main__":
        print('main start FOO value: ', shared_var.FOO)
    
        t = threading.Thread(target=func_b)
        t.start()
    
        running = True
        while running:
            if check_FOO_status():
                break
            time.sleep(3)
    
        print('main end FOO value: ', shared_var.FOO)
    

    utils.py

    import time
    import shared_var
    
    def func_b():
        print('func_b initial FOO value: ', shared_var.FOO)
        for i in range(10):
            shared_var.FOO = str(i)
            print('func_b FOO value: ', shared_var.FOO)
            time.sleep(1)
    
        shared_var.FOO = 'DONE'
        print('func_b FOO value: ', shared_var.FOO)
    

    And its output:

    main start FOO value:  INIT
    func_b initial FOO value:  INIT
    main current FOO:  INIT
    func_b FOO value:  0
    func_b FOO value:  1
    func_b FOO value:  2
    main current FOO:  2
    func_b FOO value:  3
    func_b FOO value:  4
    func_b FOO value:  5
    func_b FOO value:  6
    main current FOO:  6
    func_b FOO value:  7
    func_b FOO value:  8
    func_b FOO value:  9
    main current FOO:  9
    func_b FOO value:  DONE
    main current FOO:  DONE
    main end FOO value:  DONE