kubernetesdeploymentmicrok8s

initContainers for writable configMap


I'm writing up a deployment for graylog 5.1, however the problem I'm running into is that graylog needs to write into its graylog.conf and log4j2.xml and I'm pulling those from a config map that I've created.

The problem is they are always created as read-only and graylog cannot write into them, I read that it can be solved by create an initContainer that fix this by copying configMap into an emptyDir and then mounting that dir into the container's path: /usr/share/graylog/data/config. But for the life of me I cannot figure out how to do this.

Here's an example of the graylog-deployment.yaml:

---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: graylog
spec:
  replicas: 1
  selector:
    matchLabels:
      app: graylog
  template:
    metadata:
      labels:
        app: graylog
    spec:
      volumes:
      - name: graylog-config
        configMap:
            name: graylog-config
      initContainers:
      - name: copy-config-files
        image: alpine
        command: ["/bin/sh", "-c"]
        args:
        - |
          mkdir -p /tmp/config
          cp /usr/share/graylog/data/config/* /tmp/config/
          chmod -R 0775 /tmp/config/
        volumeMounts:
        - name: graylog-config
          mountPath: /tmp/config
      containers:
      - name: graylog
        image: graylog/graylog:5.1
        ports:
        - containerPort: 9000
        - containerPort: 1514
        - containerPort: 12201
        resources:
          requests:
            memory: "512Mi"  # Set the minimum memory required
            cpu: "250m"      # Set the minimum CPU required
          limits:
            memory: "1Gi"    # Set the maximum memory allowed
            cpu: "1"         # Set the maximum CPU allowed
        env:
        - name: GRAYLOG_MONGODB_URI
          value: "mongodb://mongodb:27017/graylog"
        - name: GRAYLOG_ELASTICSEARCH_HOSTS
          value: "http://elasticsearch:9200"
        - name: GRAYLOG_ROOT_PASSWORD_SHA2 #admin pass
          value: ****
        volumeMounts:
        - name: graylog-claim
          mountPath: /usr/share/graylog/data
        - name: graylog-config
          mountPath: /usr/share/graylog/data/config
  volumeClaimTemplates:
  - metadata:
      name: graylog-claim
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: microk8s-hostpath
      resources:
        requests:
          storage: 2Gi

However, this does not work, initContainer never finishes and I get an ouput saying that it's a read-only. I have tried many different approaches and I cannot seem to get it right.

Any help is greatly appreciated, I'm at a loss here.

I tried using emptyDir: {} approach but still the same result. I have tried with chmoding but that also doesn't work since it's a read-only files that are created: log4j2.xml and graylog.conf:

apiVersion: v1
kind: ConfigMap
metadata:
  name: graylog-config
data:
  log4j2.xml: |-
    <?xml version="1.0" encoding="UTF-8"?>
    <Configuration packages="org.graylog2.log4j" shutdownHook="disable">
        <Appenders>
            <Console name="STDOUT" target="SYSTEM_OUT">
                <PatternLayout pattern="%d %-5p: %c - %m%n"/>
            </Console>

            <!-- Internal Graylog log appender. Please do not disable. This makes internal log messages available via REST calls. -->
            <Memory name="graylog-internal-logs" bufferSize="500"/>
        </Appenders>
        <Loggers>
            <!-- Application Loggers -->
            <Logger name="org.graylog2" level="info"/>
            <Logger name="com.github.joschi.jadconfig" level="warn"/>
            <!-- Prevent DEBUG message about Lucene Expressions not found. -->
            <Logger name="org.elasticsearch.script" level="warn"/>
            <!-- Disable messages from the version check -->
            <Logger name="org.graylog2.periodical.VersionCheckThread" level="off"/>
            <!-- Silence chatty natty -->
            <Logger name="com.joestelmach.natty.Parser" level="warn"/>
            <!-- Silence Kafka log chatter -->
            <Logger name="org.graylog.shaded.kafka09.log.Log" level="warn"/>
            <Logger name="org.graylog.shaded.kafka09.log.OffsetIndex" level="warn"/>
            <Logger name="org.apache.kafka.clients.consumer.ConsumerConfig" level="warn"/>
            <!-- Silence useless session validation messages -->
            <Logger name="org.apache.shiro.session.mgt.AbstractValidatingSessionManager" level="warn"/>
            <!-- Silence Azure SDK messages -->
            <Logger name="com.azure" level="warn"/>
            <Logger name="reactor.core.publisher.Operators" level="off"/>
            <Logger name="com.azure.messaging.eventhubs.PartitionPumpManager" level="off"/>
            <Logger name="com.azure.core.amqp.implementation.ReactorReceiver" level="off"/>
            <Logger name="com.azure.core.amqp.implementation.ReactorDispatcher" level="off"/>
            <Root level="warn">
                <AppenderRef ref="STDOUT"/>
                <AppenderRef ref="graylog-internal-logs"/>
            </Root>
        </Loggers>
    </Configuration>
  graylog.conf: |-
    ############################
    # GRAYLOG CONFIGURATION FILE
    ############################
    is_master = true
    node_id_file = /usr/share/graylog/data/config/node-id
    password_secret = hs1hvm7Wi7GaChG5iDsHkvYlOnkayN4YBFeFhMosBgLNwCztbKRIZfcWA4zgx6RL9JF3I5v0mRJMNOYmF9vvuo30G2vuwAYW
    root_password_sha2 = 8c6976e5b5410415bde908bd4dee15dfb167a9c873fc4bb8a81f6f2ab448a918
    bin_dir = /usr/share/graylog/bin
    data_dir = /usr/share/graylog/data
    plugin_dir = /usr/share/graylog/plugin
    http_bind_address = 0.0.0.0:9000
    elasticsearch_hosts = http://elasticsearch:9200
    rotation_strategy = count
    elasticsearch_max_docs_per_index = 20000000
    elasticsearch_max_number_of_indices = 20
    retention_strategy = delete
    elasticsearch_shards = 4
    elasticsearch_replicas = 0
    elasticsearch_index_prefix = graylog
    allow_leading_wildcard_searches = false
    allow_highlighting = false
    elasticsearch_analyzer = standard
    output_batch_size = 500
    output_flush_interval = 1
    output_fault_count_threshold = 5
    output_fault_penalty_seconds = 30
    processbuffer_processors = 5
    outputbuffer_processors = 3
    processor_wait_strategy = blocking
    ring_size = 65536
    inputbuffer_ring_size = 65536
    inputbuffer_processors = 2
    inputbuffer_wait_strategy = blocking
    message_journal_enabled = true
    message_journal_dir = data/journal
    lb_recognition_period_seconds = 3
    mongodb_uri = mongodb://mongo/graylog
    mongodb_max_connections = 1000
    mongodb_threads_allowed_to_block_multiplier = 5
    proxied_requests_thread_pool_size = 32

Solution

  • I think you forgot to mount the emptyDir as your final directory.

    I use the following steps as shown in the below excerpt:

    1. Create a emptyDir volume (e.g., final-dir)
    2. Add it as volumeMount to both your initContainer (e.g., mounted to /dest) and your container(e.g., mounted to /some/path/your_file)
    3. Add a volume for your configMap and mount it in your initContainer (e.g., mounted to /src)
    4. Execute the cp command in your initContainer (e.g., /src/your_file /dest/your_file)

    Hope this helps.

    ---
    apiVersion: apps/v1
    kind: Deployment
    metadata:
      ..
    spec:
      ...
      template:
        ...
        spec:
          initContainers:
            - name: copy-configmap
              image: alpine
              command: ['sh', '-c', 'cp /src/your_file /dest/your_file']
              volumeMounts:
                - name: temp-dir
                  mountPath: /src
                - name: final-dir
                  mountPath: /dest
          containers:
          - name: your-container-name 
            ...
            volumeMounts:
              - name: final-dir
                mountPath: /some/path/your_file
                subPath: your_file
          volumes:
            - name: final-dir
              emptyDir: {}
            - name: temp-dir
              configMap:
                name: your-configMap-name
    ...