dockerapache-kafkakafka-rest

Kafka rest proxy log levels


I have a CLUSTER ECS with a Kafka stack installed. My kafka-rest-proxy is generating INFO log levels. I've tried adjusting it following the documentation, but I keep getting INFO logs in my Cloudwatch.

In an article, I saw that passing the variables below should change the log level.Because of the following template: log4j.properties.template

ENV KAFKA_REST_LOG4J_LOGGERS="ERROR"

ENV KAFKA_REST_LOG4J_ROOT_LOGLEVEL="ERROR"

I noticed that even with the variables declared, my file didn't change. Is this correct?

cat /etc/kafka-rest/log4j.properties 
log4j.rootLogger=INFO, stdout, file

log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c:%L)%n

log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.maxBackupIndex=10
log4j.appender.file.maxFileSize=100MB
log4j.appender.file.File=${kafka-rest.log.dir}/kafka-rest.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=[%d] %p %m (%c)%n

Cloudwatch

Cloudwatch Logs

Dockerfile

FROM confluentinc/cp-kafka-rest:7.4.0

USER root

COPY ./kafka-rest.properties /home/appuser/kafka-rest.properties
COPY ./password.properties /home/appuser/password.properties
COPY ./rest-jaas.properties /home/appuser/rest-jaas.properties

EXPOSE 80

ENV KAFKAREST_OPTS="-Djava.security.auth.login.config=/home/appuser/rest-jaas.properties"

ENV KAFKA_REST_LOG4J_LOGGERS="ERROR"

ENV KAFKA_REST_LOG4J_ROOT_LOGLEVEL="ERROR"


CMD kafka-rest-start ./kafka-rest.properties

Solution

  • Do not set CMD on your own. This will cause the templated log4j file not to be loaded.

    The run script is responsible for this

    https://github.com/confluentinc/kafka-rest-images/blob/master/kafka-rest/Dockerfile.ubi8#L79

    Best practice would also not to set USER root, and you should use env-vars to set kafka-rest.properties using its own template, not copy this file over, which may be overwritten.