etlstreamsetsdata-pipeline

Streamsets Data Collector: Replace a Field With Its Child Value


I have a data structure like this

{
  "id": 926267,
  "updated_sequence": 2304899,
  "published_at": {
      "unix": 1589574240,
      "text": "2020-05-15 21:24:00 +0100",
      "iso_8601": "2020-05-15T20:24:00Z"
  },
  "updated_at": {
      "unix": 1589574438,
      "text": "2020-05-15 21:27:18 +0100",
      "iso_8601": "2020-05-15T20:27:18Z"
  },

 }

I want to replace the updated_at field with its unix field value using Streamsets Data Collector. As far as I know, it can be done using field replacer. But I'm still didn't get it how to make a mapping expression. How can I achieve that?


Solution

  • In Field Replacer, set Fields to /rec/updated_at and New value to ${record:value('/rec/updated_at/unix')} and it will replace the value. See below.

    enter image description here

    enter image description here

    Cheers, Dash