javalogginglog4j2appender

Updating Log4J2 Appender's file path at Runtime


For a project, I have a programm that is started multiple times, and I want to log every run of it into a different File. I already have everything in place I need for this, e.g. a variable holding the intended logging location for a specific run.

The problem now is that I cannot manage to update the RollingFileAppenders of the LoggerContext correctly. It seems that they are being created correctly, and then I also seem to be able to replace the old Appender by the new one correctly, but when I actually execute some log.debug(...) or whatever, the Configuration always returns to the old Appenders for some reason. The logging then goes nowhere, not even into the file the Appender was previously set to.

I have already looked into the documentation, but this seems to suggest that what I am doing should be working.

Right now, this is what I am doing:

LoggerContext context = (LoggerContext) LogManager.getContext(false);
Configuration config = context.getConfiguration();
RollingFileAppender oldAppender = config.getAppender(appenderName);

if (oldAppender != null) {
    oldAppender.stop();

    if (!newBasePath.endsWith("\\")) {
        newBasePath += "\\";
    }
    if (!logFileName.endsWith(".log")) {
        logFileName += ".log";
    }
    String newFileName = newBasePath + logFileName;
    String newFilePattern = newBasePath + "model_%d{yyyyMMdd}-%i.log.gz";

    // Create a new RollingFileAppender with the new file path, reusing the old configuration
    RollingFileManager manager = oldAppender.getManager();
    RollingFileAppender newAppender = RollingFileAppender.newBuilder()
                .setName(appenderName)
                .withFileName(newFileName)
                .withFilePattern(newFilePattern)
                .withAppend(manager.isAppend())
                .withStrategy(manager.getRolloverStrategy())
                .withPolicy(manager.getTriggeringPolicy())
                .setLayout(oldAppender.getLayout())
                .withLocking(manager.isLocking())
                .build();
    newAppender.start();
//...

This produces a new Appender with the exact same settings as the old one, except for the write location, which is what I intended. I confirmed this in the debugger.

//remove old appender from Config and Loggers
...
    ((PropertiesConfiguration) config).removeAppender(appenderName);
    config.getLoggers().values().forEach(l -> l.removeAppender(appenderName));
    //add rebuilt (new) appender to Config and Loggers
    config.addAppender(newAppender);
    config.getLoggers().values().forEach(l -> l.addAppender(newAppender, l.getLevel(), l.getFilter()));

    context.updateLoggers();
}

This removes the old Appender from the current LoggerContext's Configuration and inserts the new one. In the Debugger, I can see that this also seems to work as intended.

This screenshot shows the Logger before changing to the new Appender (notice old log file path in the variables): Before setting new Appender

This is after executing the next few lines (notice the new log file paths):
After setting new Appender

Immediately after this, I test the Appender via some log.debug(...), but when I debug into the code, I see that the Appenders are back to the old file path:
Calling log.debug(...) immediately after updating Appender

I suspect that it somehow resorts to some default configuration that is inferred from the log4j2.properties and not affected by the changes I made.

The debug level should not be the problem, since all levels in the log4j2.properties are set to DEBUG.


Solution

  • log4j2.xml

    <?xml version="1.0" encoding="UTF-8"?>
    <Configuration status="WARN">
      <Appenders>
        <Routing name="RoutingAppender1">
          <Routes pattern="${ctx:jobId}">
            <Route>
              <RollingFile name="JobLog" fileName="logs/job-${ctx:jobId}.log"
                           filePattern="logs/job-${ctx:jobId}-%d{yyyy-MM-dd}-%i.log.gz">
                <PatternLayout>
                  <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
                </PatternLayout>
                <Policies>
                  <TimeBasedTriggeringPolicy />
                  <SizeBasedTriggeringPolicy size="10MB"/>
                </Policies>
              </RollingFile>
            </Route>
          </Routes>
        </Routing>
        <Routing name="RoutingAppender2">
          <Routes pattern="${ctx:jobId}">
            <Route>
              <RollingFile name="JobLog" fileName="logs/${ctx:jobId}/app.log"
                           filePattern="logs/${ctx:jobId}/app-%d{yyyy-MM-dd}-%i.log.gz">
                <PatternLayout>
                  <Pattern>%d %p %c{1.} [%t] %m%n</Pattern>
                </PatternLayout>
                <Policies>
                  <TimeBasedTriggeringPolicy />
                  <SizeBasedTriggeringPolicy size="10MB"/>
                </Policies>
              </RollingFile>
            </Route>
          </Routes>
        </Routing>    
      </Appenders>
    
      <Loggers>
        <Root level="info">
          <AppenderRef ref="RoutingAppender1"/>
          <AppenderRef ref="RoutingAppender2"/>      
        </Root>
      </Loggers>
    </Configuration>
    

    DemoApp.java

    package com.example;
    
    
    import org.apache.logging.log4j.LogManager;
    import org.apache.logging.log4j.Logger;
    import org.apache.logging.log4j.ThreadContext;
    
    public class DemoApp {
        private static final Logger logger = LogManager.getLogger(DemoApp.class);
        public static void executeJob(String jobId) {
            //Set jobId as a variable of ThreadContext
            ThreadContext.put("jobId", jobId);
            logger.info("Job " + jobId + " is starting...");
            // job execution logic
            ThreadContext.clearAll();  // Clear context variables to prevent contamination
        }
        public static void main(String[] args) {
            ThreadContext.put("jobId", "ZZZ");
            
            //Test English
            String msg = "Hello World!";
            System.out.println( msg );
            logger.info(msg);
            logger.warn(msg);
            logger.error(msg);
    
            executeJob("280");
    
            //Test CJK
            ThreadContext.put("jobId", "ZZZ-ZH");
            String msgCJK = "你好 世界 !";
            System.out.println( msgCJK );
            logger.info(msgCJK);
            logger.warn(msgCJK);
            logger.error(msgCJK);
    
        } //main
    
    } //class
    

    Output logs:

    RoutingAppender1

    logs
    ├── job-280.log
    ├── job-ZZZ.log
    └── job-ZZZ-ZH.log
    

    RoutingAppender2

    logs
    ├── 280
    │   └── app.log
    ├── ZZZ
    │   └── app.log
    └── ZZZ-ZH
        └── app.log