Due to application-specificity, I am trying to store either an object or a integer into a single int64 using the code below. However, the GC failed to release the resource. I am not sure where it went wrong as this example is very similar to the example in go documentation.
minimal reproducible example
package main
import (
"context"
"fmt"
"runtime"
"sync"
"sync/atomic"
)
type Chunk[T any] struct {
raw int64 // 8 bytes
}
var pool = &sync.Map{} // map[uint64]any
var lastKey = int64(0)
func NewChunkFromData[T any](data T, cancel func()) *Chunk[T] {
key := atomic.AddInt64(&lastKey, -1)
pool.Store(key, data)
line := &Chunk[T]{
raw: key,
}
runtime.AddCleanup(line, func(key int64) {
defer cancel()
pool.Delete(key)
fmt.Printf("key %d was cleaned\n", key)
}, key)
return line
}
func test() context.Context {
ctx, cancel := context.WithCancel(context.Background())
x := NewChunkFromData[[]byte]([]byte{1, 2, 3}, cancel)
_ = x
return ctx
}
func main() {
ctx := test()
runtime.GC()
runtime.GC()
runtime.Gosched()
fmt.Println("done")
<-ctx.Done()
}
expected output
done
key -1 was cleaned
actual output
done
fatal error: all goroutines are asleep - deadlock!
goroutine 1 [chan receive]:
main.main()
This happens most likely because of batching. Consider the documentation of AddCleanup:
Note that because cleanups may execute arbitrarily far into the future after an object is no longer referenced, the runtime is allowed to perform a space-saving optimization that batches objects together in a single allocation slot. The cleanup for an unreferenced object in such an allocation may never run if it always exists in the same batch as a referenced object. Typically, this batching only happens for tiny (on the order of 16 bytes or less) and pointer-free objects.
Your Chunk struct is small enough to be batched together with objects that are still alive, so it never gets garbage-collected.
If you make the struct bigger, your program works:
type Chunk[T any] struct {
raw int64
_ [24]byte
}
Playground: https://go.dev/play/p/5gAJdOwKqJE
The documentation also says "pointer-free objects", therefore without wasting too much space, you can add a pointer field to the struct:
type Chunk[T any] struct {
raw int64
_ *bool
}
Playground: https://go.dev/play/p/apWRbs7qJRK
The root cause of this issue is the same as Go GC doesn't collect dead weak pointers.