I am logging stringified json arrays from a nodejs lambda and I want to access this logs in Cloudwatch logs-insights. I use console.info for logging these arrays as I cannot use powertools due to my environment's limitations.
I am trying to access each item in the array through the jsonParse and unnest functions but it does not allow me to access every item in the array.
This is the log's structure:
[
{
'itemId': 1,
'site': 'CND'
'age': 5
},
{
'itemId': 2,
'site': 'USC',
'age': 3
},
{
'itemId': 3,
'site': 'NYC',
'age': 7
}
]
I'm hoping to access each item in the array through indexing like:
fields jsonParse(array)
| unnest array into items.0, items.1, items.2
| display items.0.itemId, items.1.site, items.2.itemId
But this does not seem to be valid for the current unnest function.
When using unnest correctly on the array, it only outputs one entry in the array and I do not even know how it determines which entry to pick:
fields jsonParse(array)
| unnest array into item
| display item
In AWS CloudWatch Logs Insights, the unnest function is typically used to flatten arrays, but it does not support directly indexing into an array like items.0, items.1, items.2. Instead, it will expand each element in the array as a separate row. Here's how you can properly extract and display the data:
First, ensure that your log message is being properly parsed as JSON:
fields @message
| parse @message '[*]' as array
| display array
If the log appears as a stringified JSON array, you need to parse it first:
fields jsonParse(@message) as array
Once parsed, you can use unnest to flatten the array:
fields jsonParse(@message) as array
| unnest array as item
| display item.itemId, item.site, item.age
Just an alternative, if your JSON structure is always fixed and you only need the first three elements, you can manually extract them (not dynamically but works for fixed-length arrays):
fields jsonParse(@message) as array
| display array[0].itemId as item1_id, array[1].site as item2_site, array[2].itemId as item3_id
This may not work in all cases due to CloudWatch’s limitations in direct array indexing.
Take note, Use filtering or manual selection if needed, but CloudWatch does not fully support array indexing like SQL.