`docker system df` vs `/system/df` (docker api endpoint)

I am writing a program in Go to get the total disk usage in GB from my docker host machine. For that, I am using the func DiskUsage() from go lib:

https://pkg.go.dev/github.com/docker/docker/client#Client.DiskUsage.

Taking a look into the code, this func is calling the docker api endpoint /system/df:

https://docs.docker.com/engine/api/v1.43/#tag/System/operation/SystemDataUsage

However, I noticed a weird behaviour when I calculate the GB using this lib vs using the command docker system df:

docker system df output:

$ docker system df
TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          223       4         21.02GB   20.7GB (98%)
Containers      6         0         0B        0B
Local Volumes   13        1         536.4MB   340.4MB (63%)
Build Cache     954       0         13.51GB   13.51GB

My Go application output:

$ go run ./cmd/main.go
Images: TOTAL (223), 17GB
Build Cache: TOTAL (954), 29GB

As you can see, we have a difference between both outputs. I would like a help to understand if I am doing something wrong with this calculation which gets data from the /system/df endpoint. Thanks :)

Go application:

package main

import (
    "context"
    "fmt"

    "github.com/docker/docker/api/types"
    "github.com/docker/docker/client"
)

func main() {
    cli, err := client.NewClientWithOpts(client.FromEnv)
    if err != nil {
        panic(err)
    }

    diskUsg, err := cli.DiskUsage(context.Background(), types.DiskUsageOptions{})
    if err != nil {
        panic(err)
    }
    b := float64(0)
    for _, ch := range diskUsg.BuildCache {
        b = b + float64(ch.Size)
    }

    b2 := float64(0)
    for _, ch := range diskUsg.Images {
        if ch.Size > ch.SharedSize {
            b2 = b2 + (float64(ch.Size) - float64(ch.SharedSize))
            continue
        }
        b2 = b2 + (float64(ch.SharedSize) - float64(ch.Size))
    }

    fmt.Printf("Images: TOTAL (%d), %2.fGB\n", len(diskUsg.Images), float64(b2)/(1<<30))
    fmt.Printf("Build Cache: TOTAL (%d), %2.fGB\n", len(diskUsg.BuildCache), float64(b)/(1<<30))
}

Solution

Based on Docker source code:

system df command: https://github.com/docker/cli/blob/v24.0.5/cli/command/system/df.go
output formatting: https://github.com/docker/cli/blob/v24.0.5/cli/command/formatter/disk_usage.go

You should be able to reproduce exactly what docker system df does using the code below:

go.mod

module 76982562-docker-system-df-vs-system-df-docker-api-endpoint

go 1.21.0

require (
    github.com/docker/cli v24.0.5+incompatible
    github.com/docker/docker v24.0.5+incompatible
)

main.go

package main

import (
    "context"
    "fmt"
    "os"

    "github.com/docker/cli/cli/command/formatter"
    "github.com/docker/docker/api/types"
    "github.com/docker/docker/client"
    "github.com/docker/go-units"
)

func main() {
    cli, err := client.NewClientWithOpts(client.FromEnv)
    if err != nil {
        panic(err)
    }

    diskUsage, err := cli.DiskUsage(context.Background(), types.DiskUsageOptions{})
    if err != nil {
        panic(err)
    }

    var bsz int64
    for _, bc := range diskUsage.BuildCache {
        if !bc.Shared {
            bsz += bc.Size
        }
    }

    fmt.Printf("Images: TOTAL (%d), %s\n", len(diskUsage.Images), units.HumanSize(float64(diskUsage.LayersSize)))
    fmt.Printf("Build Cache: TOTAL (%d), %s\n", len(diskUsage.BuildCache), units.HumanSize(float64(bsz)))
}

for images, docker library provides directly diskUsage.LayersSize to represent the total size, so you don't have to compute it yourself
for build cache, you need to exclude shared items (if !bc.Shared)

To convert sizes in the right unit, I highly recommend to use github.com/docker/go-units (e.g. units.HumanSize(float64(diskUsage.LayersSize))). This will avoid you unit conversion nightmares!