jsoncsvjqmiller

How to use miller mlr or jq to combine two JSON arrays into one array?


I used to be a jq masochist, but I discovered mlr, which seems to offer the opportunity for more (yet different) pain.

I found a nice web site that allows me to issue curl commands without API keys, etc., to give me the data I am interested in. The problem being that my chosen tool for displaying the data (Visual Data Tools' Datagraph) cannot parse the emitted JSON.

I could write something ugly using awk, sed, etc. but it would be more elegant to use something like jq or mlr to achieve this in one line.

The (x, y) pairs are in separate arrays. Is there a one-liner that will achieve the below?

A truncated example of the data that I need to transform. It seems that I could transform JSON to JSON, else JSON to CSV to achieve my goal.

{
    "btcusd": {
        "x": [
            1254700800000,
            1254787200000,
            1254873600000,
            1730419200000
        ],
        "y": [
            0.00076394,
            0.00076394,
            0.00088456,
            69513.1578978095
        ]
    },
    "xaubtc": {
        "x": [
            1254700800000,
            1254787200000,
            1254873600000,
            1719446400000
        ],
        "y": [
            1316202.8431552218,
            1359727.2037070976,
            1176008.410961382,
            0.037321851310868535
        ]
    }
}

An example of output data that Datagraph can parse (manually created):

[
   {
      "pair" : "btcusd",
      "x" : 1254700800000,
      "y" : 0.00076394
   },
   {
      "pair" : "btcusd",
      "x" : 1254787200000,
      "y" : 0.00076394
   },
   {
      "pair" : "btcusd",
      "x" : 1254873600000,
      "y" : 0.00088456
   },
   {
      "pair" : "btcusd",
      "x" : 1730419200000,
      "y" : 69513.1578978095
   },
   {
      "pair" : "xaubtc",
      "x" : 1254700800000,
      "y" : 1316202.84315522
   },
   {
      "pair" : "xaubtc",
      "x" : 1254787200000,
      "y" : 1359727.2037071
   },
   {
      "pair" : "xaubtc",
      "x" : 1254873600000,
      "y" : 1176008.41096138
   },
   {
      "pair" : "xaubtc",
      "x" : 1719446400000,
      "y" : 0.0373218513108685
   }
]

Which results in this kind of table in Datagraph

pair    y                   x
btcusd  0.00076394          1.2547008e12
btcusd  0.00076394          1.2547872e12
btcusd  0.00088456          1.2548736e12
btcusd  69513.1578978095    1.7304192e12
xaubtc  1.31620284315522e6  1.2547008e12
xaubtc  1.3597272037071e6   1.2547872e12
xaubtc  1.17600841096138e6  1.2548736e12
xaubtc  0.0373218513108686  1.7194464e12

Though this would be better (manually pretty-printed)

pair1   x1             y1                pair2   x2             y2
btcusd  1254700800000  0.00076394        xaubtc  1254700800000  1316202.8431552218
btcusd  1254787200000  0.00076394        xaubtc  1254787200000  1359727.2037070976
btcusd  1254873600000  0.00088456        xaubtc  1254873600000  1176008.410961382
btcusd  1730419200000  69513.1578978095  xaubtc  1719446400000  0.037321851310868535


Solution

  • You're looking for something like this:

    to_entries | map({pair: .key} + (.value | transpose[] | {x: .[0], y: .[1]}))