I have a docker compose that I run on my computer. Data compression works, this is shown in my logs. But for some reason there is run no automatic defragmentation. And that's why I wanted to clarify, whether etcd supports environment DEFRAGMENT_ENABLE, and is it possible to enable automatic defragmentation?
services:
etcd:
image: bitnami/etcd:3.5.20
environment:
- ALLOW_NONE_AUTHENTICATION=yes
- ETCD_ADVERTISE_CLIENT_URLS=http://test-etcd:2379
- ETCD_AUTO_COMPACTION_RETENTION=2
- ETCD_AUTO_COMPACTION_MODE=periodic
- DEFRAGMENT_ENABLE=true
- DEFRAGMENT_PERIODIC_INTERVAL=3
Below is the etcd log
{"level":"info","ts":"2025-03-22T01:23:46.712668Z","caller":"mvcc/hash.go:151","msg":"storing new hash","hash":2822278584,"revision":9,"compact-revision":-1}
{"level":"info","ts":"2025-03-22T02:23:46.717065Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T02:23:46.739152Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"22.038708ms"}
{"level":"info","ts":"2025-03-22T02:23:46.739185Z","caller":"mvcc/index.go:214","msg":"compact tree index","revision":37010}
{"level":"info","ts":"2025-03-22T02:23:47.218714Z","caller":"mvcc/kvstore_compaction.go:71","msg":"finished scheduled compaction","compact-revision":37010,"took":"479.103958ms","hash":1316096291,"current-db-size-bytes":6094848,"current-db-size":"6.1 MB","current-db-size-in-use-bytes":184320,"current-db-size-in-use":"184 kB"}
{"level":"info","ts":"2025-03-22T02:23:47.218750Z","caller":"mvcc/hash.go:151","msg":"storing new hash","hash":1316096291,"revision":37010,"compact-revision":9}
{"level":"info","ts":"2025-03-22T03:23:46.745119Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T03:23:46.753841Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"8.662593ms"}
{"level":"info","ts":"2025-03-22T04:23:46.760425Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T04:23:46.769094Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"8.626125ms"}
{"level":"info","ts":"2025-03-22T05:23:46.776632Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T05:23:46.785284Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"8.615024ms"}
{"level":"info","ts":"2025-03-22T06:23:46.791949Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T06:23:46.800617Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"8.630073ms"}
{"level":"info","ts":"2025-03-22T07:23:46.805638Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T07:23:46.814255Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"8.576452ms"}
{"level":"info","ts":"2025-03-22T08:23:46.820041Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T08:23:46.828711Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"8.627478ms"}
{"level":"info","ts":"2025-03-22T09:23:46.833890Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T09:23:46.846081Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"12.145451ms"}
{"level":"info","ts":"2025-03-22T10:23:46.851789Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T10:23:46.852438Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"611.259µs"}
{"level":"info","ts":"2025-03-22T11:23:46.857700Z","caller":"v3compactor/periodic.go:134","msg":"starting auto periodic compaction","revision":37010,"compact-period":"2h0m0s"}
{"level":"info","ts":"2025-03-22T11:23:46.867077Z","caller":"v3compactor/periodic.go:142","msg":"completed auto periodic compaction","revision":37010,"compact-period":"2h0m0s","took":"9.333535ms"}
No, bitnami/etcd currently (v3.6) doesn't provide a flag for defragmentation.
According to bitnami/etcd image README, there is no such flag as ETCD_DEFRAGMENT_*, nor anything similar.
If you want to perform automatic defragmentation, unless Bitnami integrates it in their Docker images, you can:
For reference, Etcd defragmentation is documented here:
Note that combining defragmentation with auto compaction is a great practice, but I see you already enabled the latter :)
We can create a sidecar container, under the same virtual network of the original etcd container, whose only purpose is to run etcdctl defrag to perform defragmentation on the main etcd container.
Note that this solution isn't optimal:
For more control, you would have to use an orchestrating framework such as Kubernetes, which provides CronJob abstraction (better solution) or a custom Docker image.
The following example shows how to setup a sidecar container to run defragmentation once every 10 seconds.
File compose.yaml:
name: defrag-example
services:
etcd:
image: bitnami/etcd:3.5.20
environment:
ALLOW_NONE_AUTHENTICATION: yes
ports:
- "2379:2379"
healthcheck:
test: "etcdctl endpoint health"
interval: 5s
timeout: 30s
retries: 5
networks:
etcd-net:
# Sidecar container for defragmentation
etcd-defragger:
image: bitnami/etcd:3.5.20
environment:
ALLOW_NONE_AUTHENTICATION: yes
restart: always
depends_on:
etcd:
condition: service_healthy
entrypoint: |
sh -c 'echo "Starting defragmentation at $(date)"; etcdctl defrag --endpoints=http://etcd:2379; sleep 10s'
networks:
etcd-net:
networks:
etcd-net:
driver: bridge
Run:
docker compose up
Result:
[...]
etcd-defragger-1 | Starting defragmentation at Tue Jun 10 08:26:05 UTC 2025
etcd-1 | {"level":"info","ts":"2025-06-08T19:26:06.005422Z","caller":"v3rpc/maintenance.go:92","msg":"starting defragment"}
etcd-1 | {"level":"info","ts":"2025-06-08T19:26:06.013405Z","caller":"backend/backend.go:517","msg":"defragmenting","path":"/bitnami/etcd/data/member/snap/db","current-db-size-bytes":20480,"current-db-size":"20 kB","current-db-size-in-use-bytes":16384,"current-db-size-in-use":"16 kB"}
etcd-1 | {"level":"info","ts":"2025-06-08T19:26:06.030905Z","caller":"backend/backend.go:579","msg":"finished defragmenting directory","path":"/bitnami/etcd/data/member/snap/db","current-db-size-bytes-diff":0,"current-db-size-bytes":20480,"current-db-size":"20 kB","current-db-size-in-use-bytes-diff":-4096,"current-db-size-in-use-bytes":12288,"current-db-size-in-use":"12 kB","took":"25.385613ms"}
etcd-1 | {"level":"info","ts":"2025-06-08T19:26:06.030976Z","caller":"v3rpc/maintenance.go:100","msg":"finished defragment"}
etcd-1 | {"level":"info","ts":"2025-06-08T19:26:06.030997Z","caller":"v3rpc/health.go:61","msg":"grpc service status changed","service":"","status":"SERVING"}
etcd-defragger-1 | Finished defragmenting etcd member[http://etcd:2379]
etcd-defragger-1 exited with code 0
We have to:
While trying to implement this one I stumbled upon some issues, such as the fact that crontab doesn't allow to create crontabs to users not present in /etc/passwd (such as default user with UID 1001). But I've not investigated further.
If I resolve these issues, I’ll update the answer.
Kubernetes has native support for scheduled containers, using CronJob resource (see Kuberentes Docs | CronJob and Kubernetes API | CronJob v1).
Kubernetes resource etcd-defrag-cronjob.yaml:
apiVersion: batch/v1
kind: CronJob
metadata:
name: etcd-defragger
spec:
concurrencyPolicy: Forbid
schedule: "0 0 * * *"
suspend: false
jobTemplate:
spec:
template:
spec:
containers:
- name: etcd-defrag
image: docker.io/bitnami/etcd:bitnami/etcd:latest
imagePullPolicy: "IfNotPresent"
command:
- "etcdctl"
args:
- defrag
# Here you specify the Etcd endpoint (typically you would point to the Service hostname)
- --endpoints=[]
Install with:
kubectl apply etcd-defrag-cronjob.yaml
bitnami/etcd Helm chart supports both auto compaction (based on flag ETCD_AUTO_COMPACTION_*) and defragmentation. The latter is implemented through a CronJob template.
The following example configure the defragmentation cron job to run every minute. File values.yaml:
autoCompactionMode: periodic
autoCompactionRetention: 24h
defrag:
enabled: true
cronjob:
schedule: "* * * * *"
Install with:
helm install etcd-test oci://registry-1.docker.io/bitnamicharts/etcd -f values.yaml
While the chart is installed, we can see that the CronJob spawns a Job and a Pod, which runs the defragmentation:
$ kubectl get jobs
NAME STATUS COMPLETIONS DURATION AGE
etcd-test-defrag-29159875 Complete 1/1 5s 11s
$ kubectl logs etcd-test-defrag-29159875-vkpjx
Finished defragmenting etcd member[http://etcd-test-0.etcd-test-headless.default.svc.cluster.local:2379]. took 139.565377ms
And watching the logs of the main Etcd pod, we can see that defragmentation was performed:
$ kubectl logs -f etcd-test-0
...
{"level":"info","ts":"2025-06-10T21:57:01.823712Z","caller":"v3rpc/maintenance.go:110","msg":"starting defragment"}
{"level":"info","ts":"2025-06-10T21:57:01.835577Z","caller":"backend/backend.go:522","msg":"defragmenting","path":"/bitnami/etcd/data/member/snap/db","current-db-size-bytes":20480,"current-db-size":"20 kB","current-db-size-in-use-bytes":12288,"current-db-size-in-use":"12 kB"}
{"level":"info","ts":"2025-06-10T21:57:01.859304Z","logger":"bbolt","caller":"backend/backend.go:574","msg":"Opening db file (/bitnami/etcd/data/member/snap/db) with mode -rw------- and with options: {Timeout: 0s, NoGrowSync: false, NoFreelistSync: true, PreLoadFreelist: false, FreelistType: hashmap, ReadOnly: false, MmapFlags: 8000, InitialMmapSize: 10737418240, PageSize: 0, NoSync: false, OpenFile: 0x0, Mlock: false, Logger: 0xc00040e198}"}
{"level":"info","ts":"2025-06-10T21:57:01.859597Z","logger":"bbolt","caller":"bbolt@v1.4.0/db.go:321","msg":"Opening bbolt db (/bitnami/etcd/data/member/snap/db) successfully"}
{"level":"info","ts":"2025-06-10T21:57:01.859663Z","caller":"backend/backend.go:592","msg":"finished defragmenting directory","path":"/bitnami/etcd/data/member/snap/db","current-db-size-bytes-diff":0,"current-db-size-bytes":20480,"current-db-size":"20 kB","current-db-size-in-use-bytes-diff":0,"current-db-size-in-use-bytes":12288,"current-db-size-in-use":"12 kB","took":"35.84145ms"}
{"level":"info","ts":"2025-06-10T21:57:01.859711Z","caller":"v3rpc/maintenance.go:118","msg":"finished defragment"}
...
To uninstall the Helm chart:
helm uninstall etcd-test
To see the generated resources you can run the following:
helm fetch oci://registry-1.docker.io/bitnamicharts/etcd --untar
helm template ./etcd/ --dry-run -f values.yaml > resources.yaml
And see how the CronJob is actually implemented.
Side note: I contributed to the defragmentation CronJob template on Bitnami's Helm chart :)