kubernetesweave

why kubernetes networking driver weave generating big log file


I want to ask why the Kubernetes networking driver weave container is generating a lot of logs?

The log file size is 700MB after two days.

How can I solve that?


Solution

  • Logs in kubernetes

    As it was said in comment, kubernetes is not responsible for log rotation. This is from kubernetes documentation:

    An important consideration in node-level logging is implementing log rotation, so that logs don't consume all available storage on the node. Kubernetes is not responsible for rotating logs, but rather a deployment tool should set up a solution to address that. For example, in Kubernetes clusters, deployed by the kube-up.sh script, there is a logrotate tool configured to run each hour. You can also set up a container runtime to rotate an application's logs automatically.

    As proposed option, this can be managed on container's runtime level.

    Please refer to Logging at the node level.

    Reducing logs for Weave CNI

    There are two containers in each pod. Weave itself and weave-npc (which is a network policy controller).

    By default weave's log level is set to INFO. This can be changed to WARNING to see only exceptions. This can be achieved by adding --log-level flag through the EXTRA_ARGS environment variable for the weave:

    $ kubectl edit daemonset weave-net -n kube-system
    

    So weave container part should look like:

    spec:
      containers:
      - command:
        - /home/weave/launch.sh
        env:
        - name: HOSTNAME
          valueFrom:
            fieldRef:
              apiVersion: v1
              fieldPath: spec.nodeName
        - name: EXTRA_ARGS # this was added with value below!
          value: --log-level=warning
        - name: INIT_CONTAINER
          value: "true"
        image: docker.io/weaveworks/weave-kube:2.8.1
        imagePullPolicy: IfNotPresent
        name: weave 
    

    Weave - logs level.

    A lot of logs go from Weave NPC, there's an option that allows to disable NPC. However based on documentation this is a paid option based on their documentation - cloud.weave.works

    Weave - Changing configuration options