javajsonlogginglogbackstructured-logging

How do I get JSONObject as a StructuredArgument in a LogStashEncoder logback output?


I am using logback and LogstashEncoder to write JSON format logging to the console. I am using StructuredArguments to pass the objects I want in the log from my code to the logger. However my JSONObject is not appearing as a structured object in the JSON, although it appears in the log message OK.

Here is my simple code. Note Order is just a simple POJO with a single orderId field.

        Map<String, String> user = new HashMap<>();
        user.put("firstName", "Jane");

        JSONObject userDetails = new JSONObject(user);
        
        Order order = new Order("123");

        logger.info("msg with one KV and 2 patterns {} {} {}"
            , StructuredArguments.keyValue("user", user)
            , StructuredArguments.keyValue("userDetails", userDetails)
            , StructuredArguments.keyValue("order", order)
            );

This produces the following JSON log message. I have removed some fields (log level, etc.) for brevity.

{
   "message":"msg with one KV and 2 patterns user={firstName=Jane} userDetails={\"firstName\":\"Jane\"} order=test.logging.JSONOutputExample$Order@3aa078fd",
   "level":"INFO",
   "user":{
      "firstName":"Jane"
   },
   "userDetails":{
   },
   "order":{
      "orderId":"123"
   }
}

You can see the order and user objects are nicely formatted in the both the message and the JSON object, however the org.json.JSONObject userDetails appears in the JSON log message but is empty in the full structured JSON log.

I am guessing this is perhaps something to do with a formatter or mapper? I assume toString() is used on the objects to get the inline message content (hence the order object is the fallback instance description - if I add a toString() to the Order object then I see that instead. However, for the structured JSON object something extra is going on as the Order is nicely converted to JSON, as is the user HashMap.

Here is my logback.xml file:

<configuration>
    <appender name="jsonConsoleAppender" class="ch.qos.logback.core.ConsoleAppender">
        <encoder class="net.logstash.logback.encoder.LogstashEncoder" />
    </appender>
    <root level="info">
        <appender-ref ref="jsonConsoleAppender" />
    </root>
</configuration>

And here is the dependencies in the pom.xml:

    <dependencies>
        <dependency>
            <groupId>org.json</groupId>
            <artifactId>json</artifactId>
            <version>20180130</version>
        </dependency>
        <dependency>
            <groupId>org.slf4j</groupId>
            <artifactId>slf4j-api</artifactId>
            <version>2.0.4</version>
        </dependency>
        <dependency>
            <groupId>net.logstash.logback</groupId>
            <artifactId>logstash-logback-encoder</artifactId>
            <version>7.4</version>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback</groupId>
            <artifactId>logback-core</artifactId>
            <version>1.4.7</version>
        </dependency>
        <dependency>
            <groupId>ch.qos.logback.contrib</groupId>
            <artifactId>logback-jackson</artifactId>
            <version>0.1.5</version>
        </dependency>

What am I missing? Is there a dependency missing that would map the JSONObject to actual JSON output? Should I do something different in the code to format it? Any ideas? Thanks in advance.

===============================================================

UPDATE

I updated the version of the org.json dependency to 20230618 and now the StructuredArgument output for the userDetails is not longer empty, but of the form:

   "userDetails":{
      "empty":false,
      "mapType":"java.util.HashMap"
   }

Curious..?!? Obviously the 'standard' toString() type output or something.


Solution

  • I discovered the solution to my issue. The problem I had was in a Spring Boot application, and the above question was an attempt to distill a minimal example.

    The problem is that I am using org.json.JSONObject and passing it to a StructuredArgument. When I enabled debug in the logback.xml configuration, like this ...

    <configuration debug="true">
    

    I get more output - including some errors from logback that caused log lines to not appear. From those errors I was lead to the question Could not write JSON: JsonObject; nested exception is com.fasterxml.jackson.databind.JsonMappingException: JsonObject where the suggested solution was to enable the Spring property spring.mvc.converters.preferred-json-mapper=gson. I tried this, but it made no difference. As I own the application, I changed the code to use Jackson JSON objects directly and then all was good.

    So the top tip is ... if you have logback JSON logging issues, turn on logback debug to see if there are any problems 'hidden' from you.