pythonnumpypackagefloating-point

How to dynamically change floating point precision for the calculations in the package?


I'm writing a package in python that does many array operations with different libraries (numpy, numexp, pyfftw). It consists of several utility scripts and a simulation script that parses a .ini file and runs a specific simulation. I would like to choose the precision (in the .ini file) that would be enforced for all computations for a given simulation run.

I would like to know if it is possible to implement it in the following logic (the code is not working):

config.py:

DTYPE = 'float64'

utility.py:

import numpy as np
from config import DTYPE

def create_array():
    print(DTYPE)
    arr = np.ones(10, dtype=DTYPE)
    return arr

simulation.py

from config import DTYPE
from utility import create_array

if __name__ == '__main__':
    print(DTYPE)
    precision = get_precision(ini_file)
    DTYPE = precision
    create_array()

If precision is 'float32' I would like to see the following output:

'float64'
'float32'

I have seen this issue. It's a nice workaround but since I use several libraries, monkey-patching (np.array -> np.array(..., dtype=DTYPE)) seems not ideal and I want to change precision from simulation to simulation.

I understand that my code is not working since the constant DTYPE is loaded at import time and is modified at run time.


Solution

  • The problem is that from config import DTYPE does not behave like you expect: it's not giving you a pointer to the DTYPE variable in config.py, it's actually creating a copy of it. So DTYPE in config.py, in utility.py and in simulation.py are three separate variables, and changing one will not affect the others.

    However, if you import the whole config module instead of its DTYPE variable, then it'll behave as you want:

    utility.py:

    import numpy as np
    import config
    
    def create_array():
        print(config.DTYPE)
        arr = np.ones(10, dtype=config.DTYPE)
        return arr
    

    simulation.py:

    import config
    from utility import create_array
    
    if __name__ == '__main__':
        print(config.DTYPE)
        precision = get_precision(ini_file)
        config.DTYPE = precision
        create_array()
    

    The key is to set config.DTYPE = precision, which changes the DTYPE variable in your config module, and have create_array() also use config.DTYPE, which now will be the same variable.