Dynamically creating a nested array

In some Ruby code I'm trying to convert a result from a Neo4j query into the nested form required by D3 to plot a graph, though this question is about Ruby specifically. Here's what D3 wants:

https://observablehq.com/@d3/collapsible-tree#data

i.e.

{
  id: 1
  name: 'first record'
  children: [
    {
      id: 2
      name: 'second record'
      children: []
    }
    # ... down to an arbitrary depth, with any number of children at each level.
  ]
}

What I get from Neo4j looks like this, once I have removed intermediate join records which D3 won't display:

[
  [
    {
      id: 1,
      name: 'first record'
    },
    {
      id: 2,
      name: 'second record'
    }
  ],
  [
    # ... and so on
  ]
]

I could iterate through that second data structure knowing that in each array element 0 is the parent, with children at 1, 2, 3... etc. in order. What I can't work out at present is how I might create a data structure like the first one from it, as I parse it. Somehow I'd have to go through the first one recursively to find the correct place to insert each child record.

Would anyone be able to point me in the right direction?

Edit: For anyone wanting to see a fuller example of the data:

https://gist.github.com/knirirr/527206224df3d9916d3f3c7e874b79be

An example of the desired format is given at the D3 link at the top.

Here's what I eventually came up with.

def generate_recursive_graph(start_id)
  final = {
    id: start_id,
    name: 'first node',
    children: []
  }

  data = get_data_from_neo4j(start_id)
  data.each do |row|
    next if row.length == 1

    1.upto(row.length - 1) do |index|
      # row[index - 1] is always the root of the tree as the data returned by Neo4j
      # always start from the centre of the graph. 
      recurse_row(final, row[index - 1][:id], row[index])
    end
  end

  data
end

def recurse_row(data_structure, parent_id, data)
  if data_structure[:children].nil? || data[:children].empty?
    data_structure[:children] = []
  end
  if data[:children].nil? || data[:children].empty?
    data[:children] = []
  end
  if parent_id == data_structure[:id] &&
     !data_structure[:children].collect {|x| x[:id]}.include?(data[:id])
    data_structure[:children] << data
    data_structure[:children].each do |child|
      recurse_row(child, parent_id, data)
    end
  else
    data_structure[:children].each do |child|
      recurse_row(child, parent_id, data)
    end
  end
    
  data_structure
end