I'm looking to parse an Elasticsearch query response and convert it into my own format. The response may have nested buckets and the level of nesting will be variable for each query. This is is a simplified version of the result:
{:bucket-aggregation
{:buckets
[{:key "outer_bucket"
:bucket-aggregation
{:buckets
[{:key "inner_bucket_1"
:bucket-aggregation
{:buckets
[{:key 1510657200000, :sum {:value 25}}
{:key 1510660800000, :sum {:value 50}}]}}
{:key "inner_bucket_2"
:bucket-aggregation
{:buckets
[{:key 1510657200000, :sum {:value 30}}
{:key 1510660800000, :sum {:value 35}}]}}
{:key "inner_bucket_3"
:bucket-aggregation
{:buckets
[{:key 1510657200000, :sum {:value 40}}
{:key 1510660800000, :sum {:value 45}}]}}]}}]}}
I would like to extract the :value and :key into a structure like this:
[{:key ["outer_bucket" "inner_bucket_1" 1510657200000], :value 25}
{:key ["outer_bucket" "inner_bucket_1" 1510660800000], :value 50}
{:key ["outer_bucket" "inner_bucket_2" 1510657200000], :value 30}
{:key ["outer_bucket" "inner_bucket_2" 1510660800000], :value 35}
{:key ["outer_bucket" "inner_bucket_3" 1510657200000], :value 40}
{:key ["outer_bucket" "inner_bucket_3" 1510660800000], :value 45}]
Any suggestions on how I should go about this?
edit: simplified the desired format
Here's another way to do it using clojure.walk/postwalk
that doesn't assume a fixed nesting depth i.e. it'll work with more shallow or deeply nested inputs.
(clojure.walk/postwalk
(fn [v]
(cond
;; deepest case, pull up sum value
(and (map? v) (:key v) (:sum v))
{:key [(:key v)], :value (get-in v [:sum :value])}
;; pull up unnecessary buckets map wrapper
(and (map? v) (:buckets v))
(flatten (:buckets v))
;; select outer bucket + inner buckets
(and (map? v) (:key v) (:bucket-aggregation v))
(let [outer-key (:key v)
buckets (:bucket-aggregation v)]
(map #(update % :key (fn [k] (into [outer-key] k))) buckets))
;; pass-through
:else v))
(:bucket-aggregation result))
=>
({:key ["outer_bucket" "inner_bucket_1" 1510657200000], :value 25}
{:key ["outer_bucket" "inner_bucket_1" 1510660800000], :value 50}
{:key ["outer_bucket" "inner_bucket_2" 1510657200000], :value 30}
{:key ["outer_bucket" "inner_bucket_2" 1510660800000], :value 35}
{:key ["outer_bucket" "inner_bucket_3" 1510657200000], :value 40}
{:key ["outer_bucket" "inner_bucket_3" 1510660800000], :value 45})