I am trying to release memory using gc.collect(). But I do not really understand how it works. For example:
Before I run del and gc, the used memory is 58g. After running them, the used memory is still 58. It looks that no memory has been released. I sense gc is different than shutting down python and starting again, which actually release memory.
So I am wondering how to actually release the memory so that I can load other data?
Depending on your OS, it may not take back the released memory.
You can try out malloc_trim on Linux:
import ctypes
heavy_obj = create_heavy_obj()
# do some jobs
del heavy_obj
ctypes.CDLL("libc.so.6").malloc_trim(0)