pythonpython-asynciomicropython

Object with asyncio task not destroyed when out of scope


I've noticed that objects which include an asyncio Task don't appear to be deleted when they go out of scope, and I'd like to know a better of managing these objects to avoid accumulating too many.

An example:

import asyncio

class BackgroundTask:
    def __init__(self):
        self.task = asyncio.create_task(self.heartbeat()) # line to remove
        pass
        
    async def heartbeat(self):
        while True:
            await asyncio.sleep(1)
            print("^")
    
    def __del__(self):
        print("bar deleted")

async def foo():
    bar = BackgroundTask()
    await asyncio.sleep(2)
    bar.task.cancel() # line to remove
    
async def main():
    await foo()
    print("foo has ended")
    await asyncio.sleep(2)
    print("main ended")

asyncio.run(main())

When I run this I get:

^
foo has ended
main ended
bar deleted

ie. the bar object isn't deleted until the end of the event loop, instead of just after foo ends and it goes out of scope.

If you comment out the two marked lines (ie. removed the Task) then I get the expected behaviour - the object is deleted just before foo returns into main

bar deleted
foo has ended
main ended

If foo was being called repeatedly would these objects accumulte until they started causing problems? e.g if main() was redefined as:

async def main():
    while True:
        await foo()
        print("foo has ended")
        await asyncio.sleep(2)

Is there a way of ensuring that such object are deleted when out of scope, other than manually managing how many are created?

This is actually for a MicroPython application, so if there's any difference in the answer for MicroPython I'd like to know. (I've used CPython for this demo because MicroPython doesn't have the __del__() special method)


Solution

  • I think you can trust the Python garbage collector to know what it's doing.

    If dead tasks truly accumulated without deletion, many asyncio programs would eventually crash due to uncollected objects. The asyncio package would be unusable. It's a big world out there and asyncio has been around for years now; if no one has seen a program crash due to uncollected Task objects, it's very, very likely that it's not a problem.

    To set your mind at rest I modified your program a little. You can force a garbage collection cycle at any point with the gc package from the standard library. I broke your final 2 second sleep in half and forced a GC in the middle. As you can see, the "bar" object got deleted before "main" ended, at the time of the gc.collect() call.

    I'm not an expert in how the GC works, but it seems like objects may remain undeleted for some time, perhaps until some threshold is reached that triggers a GC event.

    import gc
    import asyncio
    
    class BackgroundTask:
        def __init__(self):
            self.task = asyncio.create_task(self.heartbeat()) # line to remove
            
        async def heartbeat(self):
            while True:
                await asyncio.sleep(1)
                print("^")
        
        def __del__(self):
            print("bar deleted")
    
    async def foo():
        bar = BackgroundTask()
        await asyncio.sleep(2)
        bar.task.cancel() # line to remove
        
    async def main():
        await foo()
        print("foo has ended")
        await asyncio.sleep(1)
        print("# of objects collected", gc.collect())
        await asyncio.sleep(1)
        print("main ended")
    
    asyncio.run(main())
    

    Output:

    ^
    foo has ended
    bar deleted
    # of objects collected 15
    main ended
    

    Bottom line: go ahead and write your program. You've got better things to worry about than garbage collection.