I have a docker container running in Fargate that emits json logs to the console using log4j-layout-template or Springboot ecs format json logs.
The logs emitted look like this:
{"@timestamp":"2022-03-22T09:08:16.838Z","ecs.version":"1.2.0","log.level":"INFO","message":"Server version name: Apache Tomcat/8.5.76","process.thread.name":"main","log.logger":"org.apache.catalina.startup.VersionLoggerListener"}
{"@timestamp":"2022-03-22T09:08:16.838Z","ecs.version":"1.2.0","log.level":"INFO","message":"Server built: Feb 23 2022 17:59:11 UTC","process.thread.name":"main","log.logger":"org.apache.catalina.startup.VersionLoggerListener"}
I configure my CDK with the following:
var def = ingestGatewayTaskDefinition.addContainer(
id + "Container",
ContainerDefinitionOptions
.builder()
.image(fromEcrRepository(ecrRepository))
.memoryLimitMiB(memory)
.cpu(cpu)
.environment(environment)
.secrets(secrets)
.logging(
LogDriver.awsLogs(
AwsLogDriverProps
.builder()
.logGroup(
LogGroup.Builder
.create(this, props.getServiceName())
.logGroupName("dev/" + props.getServiceName())
.retention(RetentionDays.ONE_DAY)
.build()
)
.streamPrefix("dev/" + props.getServiceName())
//.datetimeFormat("%Y-%m-%dT%H:%M:%SZ") //??
.build()
)
)
.build()
);
But in Cloud Watch the message portion is the json and is not parsed but should be discoverable.
How do I parse these fields?
This is what is ends up looking like:
What I am looking for in Cloud Watch is this:
@timestamp | ecs.version | log.level | message | log.logger |
---|---|---|---|---|
2022-03-22T09:08:16.838Z | 1.2.0 | INFO | Server version name:... | org.apache... |
2022-03-22T09:08:16.838Z | 1.2.0 | INFO | "Server built:... | org.apache... |
There's nothing wrong with the parsing, your events are being parsed correctly.
The following query should work correctly:
fields @timestamp, @message
| filter log.level="INFO"
| sort @timestamp desc
The Log Stream UI does not show the inferred nested structure, but it's still available for querying.