I had thought that go, at the linking stage, would remove unreferenced packages and functions (and any other such dependencies) that are unused. However, in trying it, I see references to those other transitive dependencies.
I created a simple file at go-import-tester:
package main
import (
"fmt"
sub "github.com/deitch/go-import-tester-sub"
)
func main() {
fmt.Println("Hello, World!")
fmt.Println(sub.Const)
}
It imports sub.Const from the other package. The sub package defines a Const, which the main imports, but also an exported Func(), which is not referenced by main. Func() calls to github.com/containerd/containerd, giving a nice large import to work with.
package sub
import (
"log"
"github.com/containerd/containerd"
)
// Const is a constant
const Const = "Hello, Sub World!"
func Func() {
_, err := containerd.New("")
if err != nil {
log.Fatal(err)
}
}
Since my main never calls Func(), and therefore nothing from github.com/containerd/containerd, I would expect my final binary to have no dependency on github.com/containerd/containerd, let alone anything it depends upon.
Yet the build shows the following:
$ GOOS=linux go build -o importer .
$ go version -m importer
importer-linux: go1.21.5
path github.com/deitch/go-import-tester
mod github.com/deitch/go-import-tester (devel)
dep github.com/containerd/containerd v1.7.17 h1:KjNnn0+tAVQHAoaWRjmdak9WlvnFR/8rU1CHHy8Rm2A=
dep github.com/containerd/continuity v0.4.2 h1:v3y/4Yz5jwnvqPKJJ+7Wf93fyWoCB3F5EclWG023MDM=
dep github.com/containerd/fifo v1.1.0 h1:4I2mbh5stb1u6ycIABlBw9zgtlK8viPI9QkQNRQEEmY=
dep github.com/containerd/log v0.1.0 h1:TCJt7ioM2cr/tfR8GPbGf9/VRAX8D2B4PjzCpfX540I=
dep github.com/containerd/ttrpc v1.2.4 h1:eQCQK4h9dxDmpOb9QOOMh2NHTfzroH1IkmHiKZi05Oo=
dep github.com/containerd/typeurl/v2 v2.1.1 h1:3Q4Pt7i8nYwy2KmQWIw2+1hTvwTE/6w9FqcttATPO/4=
dep github.com/deitch/go-import-tester-sub v0.0.0-20240528101732-393976ab1668h1:4zY+GLhj1BdrV0NmpyM1Rx+XTU82RiLEvQd45Wv1Z18=
dep github.com/docker/go-events v0.0.0-20190806004212-e31b211e4f1c h1:+pKlWGMw7gf6bQ+oDZB4KHQFypsfjYlq/C4rfL7D3g8=
dep github.com/felixge/httpsnoop v1.0.3 h1:s/nj+GCswXYzN5v2DpNMuMQYe+0DDwt5WVCU6CWBdXk=
dep github.com/go-logr/logr v1.2.4 h1:g01GSCwiDw2xSZfjJ2/T9M+S6pFdcNtFYsp+Y43HYDQ=
dep github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
dep github.com/gogo/protobuf v1.3.2 h1:Ov1cvc58UF3b5XjBnZv7+opcTcQFZebYjWzi34vdm4Q=
dep github.com/golang/protobuf v1.5.4 h1:i7eJL8qZTpSEXOPTxNKhASYpMn+8e5Q6AdndVa1dWek=
dep github.com/google/go-cmp v0.5.9 h1:O2Tfq5qg4qc4AmwVlvv0oLiVAGB7enBSJ2x2DqQFi38=
dep github.com/google/uuid v1.3.1 h1:KjJaJ9iWZ3jOFZIf1Lqf4laDRCasjl0BCmnEGxkdLb4=
dep github.com/klauspost/compress v1.16.0 h1:iULayQNOReoYUe+1qtKOqw9CwJv3aNQu8ivo7lw1HU4=
dep github.com/moby/locker v1.0.1 h1:fOXqR41zeveg4fFODix+1Ch4mj/gT0NE1XJbp/epuBg=
dep github.com/moby/sys/mountinfo v0.6.2 h1:BzJjoreD5BMFNmD9Rus6gdd1pLuecOFPt8wC+Vygl78=
dep github.com/moby/sys/signal v0.7.0 h1:25RW3d5TnQEoKvRbEKUGay6DCQ46IxAVTT9CUMgmsSI=
dep github.com/moby/sys/user v0.1.0 h1:WmZ93f5Ux6het5iituh9x2zAG7NFY9Aqi49jjE1PaQg=
dep github.com/opencontainers/go-digest v1.0.0 h1:apOUWs51W5PlhuyGyz9FCeeBIOUDA/6nW8Oi/yOhh5U=
dep github.com/opencontainers/image-spec v1.1.0 h1:8SG7/vwALn54lVB/0yZ/MMwhFrPYtpEHQb2IpWsCzug=
dep github.com/opencontainers/runtime-spec v1.1.0 h1:HHUyrt9mwHUjtasSbXSMvs4cyFxh+Bll4AjJ9odEGpg=
dep github.com/opencontainers/selinux v1.11.0 h1:+5Zbo97w3Lbmb3PeqQtpmTkMwsW5nRI3YaLpt7tQ7oU=
dep github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
dep go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp v0.45.0 h1:x8Z78aZx8cOF0+Kkazoc7lwUNMGy0LrzEMxTm4BbTxg=
dep go.opentelemetry.io/otel v1.19.0 h1:MuS/TNf4/j4IXsZuJegVzI1cwut7Qc00344rgH7p8bs=
dep go.opentelemetry.io/otel/metric v1.19.0 h1:aTzpGtV0ar9wlV4Sna9sdJyII5jTVJEvKETPiOKwvpE=
dep go.opentelemetry.io/otel/trace v1.19.0 h1:DFVQmlVbfVeOuBRrwdtaehRrWiL1JoVs9CPIQ1Dzxpg=
dep golang.org/x/net v0.23.0 h1:7EYJ93RZ9vYSZAIb2x3lnuvqO5zneoD6IvWjuhfxjTs=
dep golang.org/x/sync v0.3.0 h1:ftCYgMx6zT/asHUrPw8BLLscYtGznsLAnjq5RH9P66E=
dep golang.org/x/sys v0.18.0 h1:DBdB3niSjOA/O0blCZBqDefyWNYveAYMNF1Wum0DYQ4=
dep golang.org/x/text v0.14.0 h1:ScX5w1eTa3QqT8oi6+ziP7dTV1S2+ALU0bI+0zXKWiQ=
dep google.golang.org/genproto v0.0.0-20230822172742-b8732ec3820d h1:VBu5YqKPv6XiJ199exd8Br+Aetz+o08F+PLMnwJQHAY=
dep google.golang.org/genproto/googleapis/rpc v0.0.0-20230822172742-b8732ec3820d h1:uvYuEyMHKNt+lT4K3bN6fGswmK8qSvcreM3BwjDh+y4=
dep google.golang.org/grpc v1.59.0 h1:Z5Iec2pjwb+LEOqzpB2MR12/eKFhDPhuqW91O+4bwUk=
dep google.golang.org/protobuf v1.33.0 h1:uNO2rsAINq/JlFpSdYEKIZ0uKD/R9cpdv0T+yoGwGmI=
build -buildmode=exe
build -compiler=gc
build CGO_ENABLED=0
build GOARCH=arm64
build GOOS=linux
build vcs=git
build vcs.revision=7672895fecf99000f9401dc8c6780523b9a59870
build vcs.time=2024-05-28T10:18:01Z
build vcs.modified=false
And sizes:
+----------------------------------------------------------------------------------------------+
| importer-linux |
+---------+---------------------------------------------------------------+--------+-----------+
| PERCENT | NAME | SIZE | TYPE |
+---------+---------------------------------------------------------------+--------+-----------+
| 13.80% | .rodata | 2.4 MB | section |
| 9.73% | .gopclntab | 1.7 MB | section |
| 9.55% | .debug_info | 1.7 MB | section |
| 6.64% | .debug_loc | 1.2 MB | section |
| 6.11% | google.golang.org/protobuf | 1.1 MB | vendor |
| 4.64% | .debug_line | 806 kB | section |
| 4.54% | .strtab | 789 kB | section |
| 4.19% | net | 728 kB | std |
| 3.86% | crypto | 670 kB | std |
| 3.57% | runtime | 621 kB | std |
| 3.43% | google.golang.org/grpc | 595 kB | vendor |
| 2.76% | github.com/containerd/containerd | 479 kB | vendor |
| 2.43% | .text | 422 kB | section |
| 1.85% | github.com/gogo/protobuf | 321 kB | vendor |
| 1.77% | .noptrdata | 308 kB | section |
| 1.58% | .debug_ranges | 275 kB | section |
| 1.47% | .symtab | 255 kB | section |
| 1.22% | .debug_frame | 212 kB | section |
| 1.10% | reflect | 190 kB | std |
| 0.96% | text/template | 166 kB | std |
| 0.95% | golang.org/x/net | 165 kB | vendor |
| 0.78% | regexp | 135 kB | std |
| 0.77% | | 133 kB | generated |
| 0.71% | math | 124 kB | std |
| 0.71% | go.opentelemetry.io/otel | 124 kB | vendor |
| 0.66% | github.com/klauspost/compress | 114 kB | vendor |
| 0.63% | time | 109 kB | std |
| 0.60% | encoding/json | 105 kB | std |
| 0.59% | .data | 102 kB | section |
| 0.58% | os | 100 kB | std |
| 0.52% | html | 90 kB | std |
| 0.41% | fmt | 71 kB | std |
| 0.38% | syscall | 66 kB | std |
| 0.36% | internal/poll | 62 kB | std |
| 0.35% | github.com/sirupsen/logrus | 62 kB | vendor |
| 0.29% | encoding/asn1 | 51 kB | std |
| 0.27% | strconv | 47 kB | std |
| 0.26% | github.com/golang/protobuf | 46 kB | vendor |
| 0.24% | sync | 41 kB | std |
| 0.23% | strings | 40 kB | std |
| 0.22% | internal/abi | 38 kB | std |
| 0.21% | mime | 37 kB | std |
| 0.19% | vendor/golang.org/x/text/unicode/norm | 32 kB | std |
| 0.18% | internal/reflectlite | 32 kB | std |
| 0.18% | github.com/google/go-cmp | 30 kB | vendor |
| 0.15% | unicode | 26 kB | std |
| 0.15% | vendor/golang.org/x/net/dns/dnsmessage | 26 kB | std |
| 0.15% | bytes | 26 kB | std |
| 0.14% | context | 24 kB | std |
| 0.13% | io | 23 kB | std |
| 0.13% | vendor/golang.org/x/net/http2/hpack | 23 kB | std |
| 0.13% | compress/flate | 22 kB | std |
| 0.12% | github.com/go-logr/logr | 21 kB | vendor |
| 0.12% | sort | 21 kB | std |
| 0.10% | vendor/golang.org/x/crypto/cryptobyte | 18 kB | std |
| 0.10% | bufio | 17 kB | std |
| 0.09% | vendor/golang.org/x/net/idna | 16 kB | std |
| 0.09% | log | 16 kB | std |
| 0.09% | go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp | 15 kB | vendor |
| 0.07% | internal/bisect | 12 kB | std |
| 0.06% | github.com/go-logr/stdr | 9.9 kB | vendor |
| 0.05% | github.com/docker/go-events | 9.5 kB | vendor |
| 0.05% | github.com/containerd/ttrpc | 8.9 kB | vendor |
| 0.05% | text/tabwriter | 8.9 kB | std |
| 0.05% | vendor/golang.org/x/net/http/httpproxy | 8.9 kB | std |
| 0.05% | google.golang.org/genproto | 8.3 kB | vendor |
| 0.04% | .itablink | 7.5 kB | section |
| 0.04% | path | 7.3 kB | std |
| 0.04% | internal/godebug | 6.6 kB | std |
| 0.04% | internal/fmtsort | 6.6 kB | std |
| 0.04% | container/list | 6.4 kB | std |
| 0.04% | vendor/golang.org/x/crypto/chacha20 | 6.2 kB | std |
| 0.03% | encoding/base64 | 5.8 kB | std |
| 0.03% | github.com/containerd/typeurl/v2 | 5.7 kB | vendor |
| 0.03% | compress/gzip | 5.1 kB | std |
| 0.03% | hash/crc32 | 4.7 kB | std |
| 0.03% | github.com/google/uuid | 4.5 kB | vendor |
| 0.02% | vendor/golang.org/x/crypto/chacha20poly1305 | 4.0 kB | std |
| 0.02% | .go.buildinfo | 3.8 kB | section |
| 0.02% | internal/singleflight | 3.7 kB | std |
| 0.02% | golang.org/x/sys | 3.6 kB | vendor |
| 0.02% | internal/bytealg | 3.1 kB | std |
| 0.02% | encoding/pem | 3.1 kB | std |
| 0.02% | errors | 2.9 kB | std |
| 0.02% | internal/cpu | 2.8 kB | std |
| 0.01% | vendor/golang.org/x/crypto/internal/poly1305 | 2.5 kB | std |
| 0.01% | internal/intern | 2.5 kB | std |
| 0.01% | vendor/golang.org/x/text/unicode/bidi | 2.3 kB | std |
| 0.01% | golang.org/x/text | 2.1 kB | vendor |
| 0.01% | archive/tar | 2.0 kB | std |
| 0.01% | vendor/golang.org/x/crypto/hkdf | 1.9 kB | std |
| 0.01% | vendor/golang.org/x/net/http/httpguts | 1.8 kB | std |
| 0.01% | internal/testlog | 1.7 kB | std |
| 0.01% | vendor/golang.org/x/text/secure/bidirule | 1.6 kB | std |
| 0.01% | internal/syscall/unix | 1.5 kB | std |
| 0.01% | github.com/opencontainers/go-digest | 1.0 kB | vendor |
| 0.01% | github.com/containerd/log | 984 B | vendor |
| 0.00% | github.com/opencontainers/selinux | 760 B | vendor |
| 0.00% | internal/itoa | 644 B | std |
| 0.00% | hash/fnv | 360 B | std |
| 0.00% | github.com/opencontainers/runtime-spec | 356 B | vendor |
| 0.00% | github.com/opencontainers/image-spec | 356 B | vendor |
| 0.00% | go/token | 324 B | std |
| 0.00% | .debug_abbrev | 309 B | section |
| 0.00% | main | 308 B | main |
| 0.00% | github.com/moby/sys/user | 308 B | vendor |
| 0.00% | database/sql/driver | 292 B | std |
| 0.00% | .shstrtab | 263 B | section |
| 0.00% | encoding/binary | 244 B | std |
| 0.00% | github.com/moby/sys/signal | 196 B | vendor |
| 0.00% | .note.go.buildid | 100 B | section |
| 0.00% | .debug_gdb_scripts | 67 B | section |
| 0.00% | github.com/containerd/fifo | 0 B | vendor |
| 0.00% | github.com/moby/sys/mountinfo | 0 B | vendor |
| 0.00% | github.com/deitch/go-import-tester-sub | 0 B | vendor |
| 0.00% | github.com/felixge/httpsnoop | 0 B | vendor |
| 0.00% | golang.org/x/sync | 0 B | vendor |
| 0.00% | github.com/deitch/go-import-tester | 0 B | vendor |
| 0.00% | github.com/moby/locker | 0 B | vendor |
| 0.00% | github.com/containerd/continuity | 0 B | vendor |
+---------+---------------------------------------------------------------+--------+-----------+
| 99.21% | KNOWN | 17 MB | |
| 100% | TOTAL | 17 MB | |
+---------+---------------------------------------------------------------+--------+-----------+%
That is a lot of imported packages, which do add up to size, for something that isn't used.
It makes sense that at initial compile time, the go compiler would have to compile sub, which depends on containerd, so it would have to create the compiled static library for it. However, when linking, should it not be able to look and say, "you depend on Const but not Func, so I can get rid of everything else?
What am I misunderstanding?
EDIT:
I also compiled it with referencing sub.Func(), and the size of the binary does grow slightly, as do the sizes of the imported modules. E.g. containerd grows from 479KB to 516KB, github.com/google/go-cmp grows from 30KB, to 31KB, so clearly it matters. But why does it have them in there at all?
The linker can probably prove that sub.Func is unreferenced and eliminate that.
But you import github.com/deitch/go-import-tester-sub which imports github.com/containerd/containerd which has an init function, which is expected to run. This can't be skipped, since other inits might depend on that package to be successfully initialized - and in the end the linker can't reason what side effects happen. Also, since the reflect package is picked up, all exported symbols are reachable.
For example if you have a module go-import with a main:
package main
import (
"fmt"
_ "go-import/pkg/sub1"
)
func main() {
fmt.Println("Hello from main")
}
pkg/sub1/sub1.go:
package sub1
import (
_ "go-import/pkg/sub2"
)
and pkg/sub2/sub2.go:
package sub2
import "fmt"
func init() {
fmt.Println("Init from sub2")
}
var World = Hello()
func Hello() *struct{} {
fmt.Println("Hello from sub2")
return nil
}
You'll expect go run to output:
Hello from sub2
Init from sub2
Hello from main
Event though you don't use anything from sub2 directly, the Go runtime does.