Limit API logs to Fewer Environments

I am in the process of sending my API logs from MuleSoft CloudHub to Datadog. I am able to send the logs by making the necessary changes to the application's log4j2.xml. We have several MuleSoft environments like DEV, TEST, UAT and Prod. I don't want to send the logs from all environments to Datadog and would like to limit to only UAT and Prod. Where can I do that filtering? Is there any specific query parameter for the datadog intake url (https://http-intake.logs.datadoghq.eu/api/v2/logs) where I can specify if the logs need to be sent for an environment or not?

For example, using the below Datadog API, I can send the logs.

curl --location 'https://http-intake.logs.datadoghq.eu/api/v2/logs?host=localhost&ddsource=Mulesoft&service=TestApp&ddtags=test' \
--header 'DD-API-KEY: a37607ff-112a-4dde-b4b3-ded433f2e2c3' \
--header 'Content-Type: application/json' \
--data '{
    "Test1": "Test-08JUL24"
}'

I did see this: Is there a way to disable datadog's log collection for a development environment?, but haven't succeeded yet

Solution

There are probably multiple ways to achieve it. One of the ways is to set the level of your AppenderRef under AsyncRoot dynamically using runtime properties. Suppose you want to send logs to datadog for prod and for qa you want to send it to some other appender you can configure the log4j2.xml something like the following

<?xml version="1.0" encoding="utf-8"?>
<Configuration>
    <Appenders>
        <Http name="DataDog">
            <!-- Appender details -->  
        </Http>
        <SomeOtherAppender name="NonProd">
              <!-- Appender details -->  
        </SomeOtherAppender>
    </Appenders>

    <Loggers>
        <!-- other loggers -->  
        <AsyncRoot level="INFO">
            <AppenderRef ref="DataDog" level="${sys:logging.appenders.datadog.level:-trace}" /> 
            <AppenderRef ref="NonProd" level="${sys:logging.appenders.nonprod.level:-trace}" />
        </AsyncRoot>
    </Loggers>
</Configuration>

What this will do is it will look for runtime property logging.appenders.datadog.level while setting up datadog appender, and if not found then it will set to trace (because of expression ${sys:logging.appenders.nonprod.level:-trace}). And same for the other logger. In other words the default behavior of these apenders will be to log all levels from info to trace.

Now you set the the logging.appenders.datadog.level=off for non prod envs while deploying.

NOTE: The parameter needs to be set as runtime propert only and will not work if you put it in the application's yaml files. It is because the loggers initializes before the application's yaml is read.