java tomcat logging log4j embedded-tomcat-8

Embedded Tomcat using log4j for logging

I'm using embedded Tomcat 8.5.4, i.e.,

<dependency>
    <groupId>org.apache.tomcat.embed</groupId>
    <artifactId>tomcat-embed-core</artifactId>
    <version>8.5.4</version>
</dependency>

The implementation is working perfectly (Tomcat works like a charm), the only thing that's bothering me, is that the embedded Tomcat logs on System.out. Internally within my application I'm using log4j for logging, so this leads to the following logging mixture (and not logging of Tomcat to any file):

...
2017-07-30 17:57:54 DEBUG EmbeddedTomcat:136 - Binding servlet 'sample' to path '/sample/*'.
Jul 30, 2017 5:57:54 PM org.apache.coyote.AbstractProtocol init
INFO: Initializing ProtocolHandler ["http-nio-15000"]
Jul 30, 2017 5:57:54 PM org.apache.tomcat.util.net.NioSelectorPool getSharedSelector
INFO: Using a shared selector for servlet write/read
Jul 30, 2017 5:57:54 PM org.apache.catalina.core.StandardService startInternal
INFO: Starting service Tomcat
Jul 30, 2017 5:57:54 PM org.apache.catalina.core.StandardEngine startInternal
INFO: Starting Servlet Engine: Apache Tomcat/8.5.4
Jul 30, 2017 5:57:54 PM org.apache.coyote.AbstractProtocol start
INFO: Starting ProtocolHandler [http-nio-15000]
2017-07-30 17:57:54 INFO  EmbeddedTomcat:80 - Successfully started Tomcat on port 15000 (base: null, url: http://localhost:15000).
...

In this snippet the first and the last line (after and before the ...) are logged by my application using log4j and the configuration of log4j (which writes to a file and the System.out). Nevertheless, the middle part (logging of Tomcat) is handled by the Embedded Tomcat and I have no clue, how to get Tomcat to use the available log4j (and it's configuration).

I tried to add the following dependency (a 8.5.4 version is not available on Maven Repository or Maven Central), but without any success.

<dependency>
    <groupId>org.apache.tomcat.embed</groupId>
    <artifactId>tomcat-embed-logging-log4j</artifactId>
    <version>8.5.2</version>
</dependency>

Does anyone know how to get the Embedded Tomcat to log using log4j (Version 1, I'm not using log4j2)?

I looked at/tried the following StackOverflow answers:

https://tomcat.apache.org/tomcat-8.0-doc/logging.html So I looked at the documentation, which mentions log4j as logging framework. It mentions tomcat-juli-adapters.jar, which I could not find for the embedded version (is it the same as the "normal" Tomcat?). How would I do that programmatically, i.e., within my Embedded Tomcat implementation.
Tomcat Logging using log4j? That's not really the problem I'm having, it's not based on the Embedded Tomcat, the version is pretty old, and I'm actually using log4j not System.out.
Embedded Tomcat logging over logback / sl4j This question actually deals with logback and the author mentions that I found some info about using a standalone tomcat with log4j, but the standalone is different, I see the author is using similar dependencies, but not sure if there was ever a solution.
How to enable embedded tomcat logging First I thought his may be the solution, but it all just deals with an additional logging for the Embedded Tomcat. I want the Embedded Tomcat to use the applications log4j, so one log4j.properties file, which defines how everything is logged.
Logging in Embedded Tomcat I'm not sure why this answer is even marked as correct, but this is just an explanation how Tomcat writes the catalina.out file and not how the logging of the Embedded Tomcat works.

Solution

It took me a while, but after I got the sources of the 8.5.4 implementation, I realized that the juli logging implementation is added in the core jar.

Version <= 8.5.2

When working with Tomcat 8.5.2, use the following jars:

1. tomcat-embed-logging-log4j-8.5.2.jar

2. tomcat-embed-core-8.5.2.jar

Do not include the tomcat-embed-logging-juli-8.5.2.jar, despite what some online documentation might suggest.

With this configuration, Tomcat 8.5.2 works with log4j out of the box, requiring no additional setup. This approach provides a straightforward solution for integrating log4j with Tomcat 8.5.2.

Version > 8.5.2

When using newer versions of embedded Tomcat (8.5.4 and above), the LogFactory class is already included in the core jar. If you add an older tomcat-embed-logging-log4j-8.5.2.jar to your classpath, you will end up with (2) two LogFactory implementations:

1. Core-LogFactory: Provided by Tomcat core, which loads DirectJDKLog.

2. Log4j-LogFactory: Provided by the additional log4j jar.

This duplication causes issues because:

The classloader typically picks the Core-LogFactory due to its proximity in the classpath.
Having multiple versions of the same class in the classpath is generally considered bad practice and can lead to unpredictable behavior.

To resolve this, instead of using the older tomcat-embed-logging-log4j-8.5.2.jar, it's recommended to use the ServiceLoader approach implemented in the Core-LogFactory of newer Tomcat versions. This approach avoids class duplication and potential conflicts, providing a more reliable and maintainable logging setup.

private LogFactory() {
    // Look via a ServiceLoader for a Log implementation that has a
    // constructor taking the String name.
    ServiceLoader<Log> logLoader = ServiceLoader.load(Log.class);
    Constructor<? extends Log> m=null;
    for (Log log: logLoader) {
        Class<? extends Log> c=log.getClass();
        try {
            m=c.getConstructor(String.class);
            break;
        }
        catch (NoSuchMethodException | SecurityException e) {
            throw new Error(e);
        }
    }
    discoveredLogConstructor=m;
}

To do so, I added the file org.apache.juli.logging.Log in the META-INF/services folder (within my jar/sources) and added the fully qualified name of my "own" Log implementation, i.e., net.meisen.tomcat.logging.Log4jLog, which is as following:

package net.meisen.tomcat.logging;

import org.apache.juli.logging.Log;
import org.apache.log4j.Logger;

public class Log4jLog implements Log {
    private final Logger logger;

    // this constructor is important, otherwise the ServiceLoader cannot start
    public Log4jLog() {
        logger = Logger.getLogger(Log4jLog.class);
    }

    // this constructor is needed by the LogFactory implementation
    public Log4jLog(final String name) {
        logger = Logger.getLogger(name);
    }

    // now we have to implement the `Log` interface
    @Override
    public boolean isFatalEnabled() {
        return true;
    }

    // ... more isLevelEnabled()

    @Override
    public boolean isTraceEnabled() {
        return logger.isTraceEnabled();
    }

    // ... and also all the fatal(...) - trace(...) methods

    @Override
    public void fatal(final Object msg) {
        logger.fatal(msg);
    }

    @Override
    public void fatal(final Object msg, final Throwable throwable) {
        logger.fatal(msg, throwable);
    }
}

And et voilà, here is the final result:

2017-07-31 19:27:04 TRACE EmbeddedTomcat:48 - Initializing Tomcat on port 15000 (base: null)...
2017-07-31 19:27:33 INFO  Http11NioProtocol:69 - Initializing ProtocolHandler ["http-nio-15000"]
2017-07-31 19:27:33 INFO  NioSelectorPool:69 - Using a shared selector for servlet write/read
2017-07-31 19:27:33 INFO  StandardService:69 - Starting service [Tomcat]
2017-07-31 19:27:33 INFO  StandardEngine:69 - Starting Servlet Engine: Apache Tomcat/8.5.19
2017-07-31 19:27:34 WARN  SessionIdGeneratorBase:79 - Creation of SecureRandom instance for session ID generation using [SHA1PRNG] took [170] milliseconds.
2017-07-31 19:27:34 INFO  Http11NioProtocol:69 - Starting ProtocolHandler ["http-nio-15000"]
2017-07-31 19:27:34 INFO  EmbeddedTomcat:80 - Successfully started Tomcat on port 15000 (base: null, url: http://localhost:15000).

Appendix:

Here are some links that helped me to figure out the ServiceLoader stuff and why I decided pretty fast not to have the same class in the same package in different jars in my project: