I have a CLUSTER ECS with a Kafka stack installed. My kafka-rest-proxy is generating INFO log levels. I've tried adjusting it following the documentation, but I keep getting INFO logs in my Cloudwatch.
In an article, I saw that passing the variables below should change the log level.Because of the following template: log4j.properties.template
ENV KAFKA_REST_LOG4J_LOGGERS="ERROR"
ENV KAFKA_REST_LOG4J_ROOT_LOGLEVEL="ERROR"
I noticed that even with the variables declared, my file didn't change. Is this correct?
cat /etc/kafka-rest/log4j.properties
log4j.rootLogger=INFO, stdout, file
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.layout=org.apache.log4j.PatternLayout
log4j.appender.stdout.layout.ConversionPattern=[%d] %p %m (%c:%L)%n
log4j.appender.file=org.apache.log4j.RollingFileAppender
log4j.appender.file.maxBackupIndex=10
log4j.appender.file.maxFileSize=100MB
log4j.appender.file.File=${kafka-rest.log.dir}/kafka-rest.log
log4j.appender.file.layout=org.apache.log4j.PatternLayout
log4j.appender.file.layout.ConversionPattern=[%d] %p %m (%c)%n
Cloudwatch
Dockerfile
FROM confluentinc/cp-kafka-rest:7.4.0
USER root
COPY ./kafka-rest.properties /home/appuser/kafka-rest.properties
COPY ./password.properties /home/appuser/password.properties
COPY ./rest-jaas.properties /home/appuser/rest-jaas.properties
EXPOSE 80
ENV KAFKAREST_OPTS="-Djava.security.auth.login.config=/home/appuser/rest-jaas.properties"
ENV KAFKA_REST_LOG4J_LOGGERS="ERROR"
ENV KAFKA_REST_LOG4J_ROOT_LOGLEVEL="ERROR"
CMD kafka-rest-start ./kafka-rest.properties
Do not set CMD
on your own. This will cause the templated log4j file not to be loaded.
The run
script is responsible for this
https://github.com/confluentinc/kafka-rest-images/blob/master/kafka-rest/Dockerfile.ubi8#L79
Best practice would also not to set USER root
, and you should use env-vars to set kafka-rest.properties
using its own template, not copy this file over, which may be overwritten.