pythonwindowsloggingfile-permissionstemporary-directory

Why can't my file handler's file within a temporary directory be cleaned up on Windows?


I have some code which runs fine on unix systems, but not on Windows. I'd like to make it cross-platform, but am banging my head against the wall. The minimal repro is as follows:

File 1: foo.py

import os
import sys
import logging

logging.basicConfig(level=logging.INFO, stream=sys.stdout)
logger = logging.getLogger('foo')

def main(dir):
    logger.addHandler(logging.FileHandler(
        os.path.join(dir, 'temporary.log')
    ))

    logger.info("Hello, world!")

File 2: main.py

from foo import main

import tempfile

if __name__ == "__main__":
    with tempfile.TemporaryDirectory("test") as tmp:
        main(tmp)

What I'd expect is that the temporary directory would be created, a file would be created within that to which logs would be emitted, and then both would be cleaned up when tmp goes out of scope.

Instead, Windows provides an error:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: '...'

I've tried changing the mode of the FileHandler away from append mode, I've tried manually cleaning up the files and directories, I've tried delaying the file from being created until its logged to and cranked up the log level, and I've even tried instantiating the logger inside foo.main in hopes that no references to the handler would be persist -- regardless, I still see this error.

How can I fix this?


Solution

  • You need to close the handler, which closes the file. Then the deletion of the temporary directory should work. I made changes as follows:

    # foo.py
    import os
    import sys
    import logging
    
    logging.basicConfig(level=logging.INFO, stream=sys.stdout)
    logger = logging.getLogger('foo')
    
    def main(dir):
        h = logging.FileHandler(os.path.join(dir, 'temporary.log'))
        logger.addHandler(h)
        logger.info("Hello, world!")
        logger.removeHandler(h)
        return h
    

    and

    # main.py
    from foo import main
    
    import tempfile
    
    if __name__ == "__main__":
        with tempfile.TemporaryDirectory("test") as tmp:
            print('Using temp dir %s' % tmp)
            h = main(tmp)
            h.close()
    

    Following which, it seems to work:

    ~> python3 c:\temp\main.py
    Using temp dir C:\Users\Vinay\AppData\Local\Temp\tmp60qirkhutest
    INFO:foo:Hello, world!
    
    ~> dir AppData\Local\Temp\tmp60qirkhutest
     Volume in drive C has no label.
     Volume Serial Number is D195-0C0D
    
     Directory of C:\Users\Vinay\AppData\Local\Temp
    
    File Not Found
    

    If you comment out the h.close() line, it fails as before.