go pointers interface garbage-collection weak-references

Go GC doesn't collect dead weak pointers

I am trying to use weak pointers introduced in go1.24 to build a system in which weak pointers can be passed as interfaces. The problem is that weak.Pointer doesn't follow the same interface as the strong pointer it wraps. To fix this I create a wrapper which holds a weak pointer and implements the interface of the original pointer, it passes calls to the referenced object or returns an error if it was already GC-ed.

Here is the code:

package main

import (
    "errors"
    "fmt"
    "runtime"
    "runtime/debug"
    "weak"
)

// Job is the interface your layers all share.
type Job interface {
    Execute() error
}

// ErrInvalidJob signals the underlying *MyJob has been GC’d.
var ErrInvalidJob = errors.New("job invalid (GC’ed)")

// MyJob is your concrete implementation.
type MyJob struct {
    id int
}

func (j *MyJob) Execute() error {
    fmt.Printf("running MyJob %d\n", j.id)
    return nil
}

// WeakMyJob wraps a *MyJob into a Job by using a weak.Pointer.
type WeakMyJob struct {
    wp weak.Pointer[MyJob]
}

func NewWeakMyJob(job *MyJob) *WeakMyJob {
    wp := weak.Make(job)

    runtime.AddCleanup(job, func(jobID int) {
        fmt.Printf("cleanup: MyJob %d finalized\n", jobID)
    }, job.id)

    return &WeakMyJob{wp: wp}
}

func (w *WeakMyJob) Execute() error {
    if real := w.wp.Value(); real != nil {
        return real.Execute()
    }
    return ErrInvalidJob
}

// Manager only holds the Job interface.
type Manager struct {
    jobs []Job
}

func NewManager() *Manager {
    return &Manager{jobs: make([]Job, 0)}
}

func (m *Manager) Add(job Job) {
    m.jobs = append(m.jobs, job)
}

func (m *Manager) RunAll() {
    updatedJobs := m.jobs[:0]
    for i, job := range m.jobs {
        err := job.Execute()
        if errors.Is(err, ErrInvalidJob) {
            fmt.Printf("job[%d] removed: %v\n", i, err)
            continue
        }
        if err != nil {
            fmt.Printf("job[%d] failed: %v\n", i, err)
        }
        updatedJobs = append(updatedJobs, job)
    }
    m.jobs = updatedJobs
}

func main() {
    mgr := NewManager()

    for i := range 10 {
        j := &MyJob{id: i}
        mgr.Add(NewWeakMyJob(j))
        j = nil
    }

    fmt.Println("=== Before GC ===")
    mgr.RunAll()

    runtime.GC()
    debug.FreeOSMemory()

    fmt.Println("=== After GC ===")
    mgr.RunAll()
}

In this code:

Job is the interface I implement
MyJob is an actual implementation of Job
WeakMyJob wraps a weak pointer to MyJob and implements the Job interface. This is what then passed to manager.
Manager holds a slice of Job (being WeakMyJob under the hood) objects and tries to call 1 by 1 in RunAll method. This way I am trying to simulate a struct which works with some objects which are supposed to be GCed without its knowledge and it must handle it gracefully by removing such objects from its slice
In main jobs are created as pointers, wrapped with MyWeakJob and added to manager. After the for loop finishes all jobs aren't accessible from anywhere except from its weak pointer and, thus, must be GCed, as I see it.

When I run it locally (Macbook Pro M3) the output isn't consistent. Sometimes it is

=== Before GC ===
running MyJob 0
running MyJob 1
running MyJob 2
running MyJob 3
running MyJob 4
cleanup: MyJob 0 finalized
=== After GC ===
job[0] removed: job invalid (GC’ed)
running MyJob 1
running MyJob 2
running MyJob 3
running MyJob 4

And other times

=== Before GC ===
running MyJob 0
running MyJob 1
running MyJob 2
running MyJob 3
running MyJob 4
=== After GC ===
running MyJob 0
running MyJob 1
running MyJob 2
running MyJob 3
running MyJob 4

Not only they change from run to run but also only 1 job is GCed. I ran it few dozens times and only got this 2 outputs. I also tried running runtime.GC() in a loop but it achieved nothing.

I tried extracting job creation to a separate function like below and it also hasn't changed anything:

func createJobs(mgr *Manager) {
    for i := range 5 {
        j := &MyJob{id: i}
        mgr.Add(NewWeakMyJob(j))
        j = nil
    }
}

func main() {
    mgr := NewManager()
    createJobs(mgr)

    fmt.Println("=== Before GC ===")
    mgr.RunAll()

    runtime.GC()
    debug.FreeOSMemory()

    fmt.Println("=== After GC ===")
    mgr.RunAll()
}

Escape anaylis doesn't contain anything unexpected it seems:

./scratch_68.go:24:7: j does not escape
./scratch_68.go:25:12: ... argument does not escape
./scratch_68.go:25:36: j.id escapes to heap
/opt/homebrew/opt/go/libexec/src/runtime/mcleanup.go:69:52: moved to heap: runtime.arg
/opt/homebrew/opt/go/libexec/src/runtime/mcleanup.go:75:9: "runtime.AddCleanup: ptr is nil" escapes to heap
/opt/homebrew/opt/go/libexec/src/runtime/mcleanup.go:80:24: runtime.arg does not escape
/opt/homebrew/opt/go/libexec/src/runtime/mcleanup.go:82:10: "runtime.AddCleanup: ptr is equal to arg, cleanup will never run" escapes to heap
/opt/homebrew/opt/go/libexec/src/runtime/mcleanup.go:87:9: "runtime.AddCleanup: ptr is arena-allocated" escapes to heap
/opt/homebrew/opt/go/libexec/src/runtime/mcleanup.go:96:8: func literal escapes to heap
/opt/homebrew/opt/go/libexec/src/runtime/mcleanup.go:110:9: "runtime.AddCleanup: ptr not in allocated block" escapes to heap
./scratch_68.go:38:13: ... argument does not escape
./scratch_68.go:38:47: jobID escapes to heap
./scratch_68.go:34:19: leaking param: job
./scratch_68.go:37:26: func literal escapes to heap
./scratch_68.go:41:9: &WeakMyJob{...} escapes to heap
./scratch_68.go:44:7: leaking param content: w
./scratch_68.go:57:9: &Manager{...} escapes to heap
./scratch_68.go:57:28: make([]Job, 0) escapes to heap
./scratch_68.go:60:7: leaking param content: m
./scratch_68.go:60:23: leaking param: job
./scratch_68.go:64:7: leaking param content: m
./scratch_68.go:69:14: ... argument does not escape
./scratch_68.go:69:40: i escapes to heap
./scratch_68.go:73:14: ... argument does not escape
./scratch_68.go:73:39: i escapes to heap
./scratch_68.go:80:17: leaking param content: mgr
./scratch_68.go:82:8: &MyJob{...} escapes to heap
./scratch_68.go:92:13: ... argument does not escape
./scratch_68.go:92:14: "=== Before GC ===" escapes to heap
./scratch_68.go:98:13: ... argument does not escape
./scratch_68.go:98:14: "=== After GC ===" escapes to heap

So, my question is why does go GC fail to collect my objects? Or is it Cleanup not geting run, why?

Solution

There is no guarantee that it will be collected. As per the documentation:

Pointer.Value is not guaranteed to eventually return nil. Pointer.Value may return nil as soon as the object becomes unreachable. [...]

Note that because Pointer.Value is not guaranteed to eventually return nil, even after an object is no longer referenced, the runtime is allowed to perform a space-saving optimization that batches objects together in a single allocation slot. The weak pointer for an unreferenced object in such an allocation may never become nil if it always exists in the same batch as a referenced object. Typically, this batching only happens for tiny (on the order of 16 bytes or less) and pointer-free objects.

Since you wrapped your weak pointer in a small struct, it's possible the runtime has batched it, therefore you don't see it getting garbage collected.

But the key point here is another: the documentation uses the term MAY in "may return nil as soon as". The typical understanding of these verbs (may, should, must) in formal specifications is outlined here, in short something described as "may" is optional.

So, you should not rely on wp.Value != nil to implement deterministic program logic.