nextflow

How to flatten nested tuple within in a Nextflow channel element


Given a nextflow channel of the following form:

original_ch = Channel.from([
   ['a', [[1, 2, 3], [4, 5, 6]]],
   ['b', [[7, 8, 9], [10, 11, 12]],
])

I would like to transform it to match the following:

transformed_ch = Channel.from([
    ['a', [1, 2, 3, 4, 5, 6]],
    ['b', [7, 8, 9, 10, 11, 12]],
])

How do I do that?

I have been trying (and failing) to find some combination of the map() operator with some way to manipulate a channel element. Example of what I tried:

original_ch.map{ [it[0], flatten(it[1])] }.view()

yields the following result:

[a, nextflow.extension.OpCall@5b09c124]
[b, nextflow.extension.OpCall@6a18fcee]

while trying:

original_ch.map{ [it[0], it[1].flatten()] }.view()

yields the desired result:

[a, [1, 2, 3, 4, 5, 6]]
[b, [7, 8, 9, 10, 11, 12]]

nextflow documentation was not too helpful with this experiment.

Based on its title, I was hoping that the post how to flatten the second element of the input tuple in a process in nextflow might help, but the answers there apparently do not quite match my use case. Please help - thank you!


Solution

  • One solution is to use the flatten method in Groovy to flatten the collection of values, for example:

    workflow {
        original_ch = Channel.from([
           ['a', [[1, 2, 3], [4, 5, 6]]],
           ['b', [[7, 8, 9], [10, 11, 12]]],
        ])
    
        transformed_ch = original_ch.map { key, vals -> tuple( key, vals.flatten() ) }
    
        transformed_ch.view()
    }
    

    Results:

    $ nextflow run main.nf 
    
     N E X T F L O W   ~  version 24.10.0
    
    Launching `main.nf` [boring_liskov] DSL2 - revision: 6e06b67b1a
    
    [a, [1, 2, 3, 4, 5, 6]]
    [b, [7, 8, 9, 10, 11, 12]]
    

    As you have discovered, this is not the same as Nextflow's flatten operator, which is intended to flatten each item (list or collection) in a source channel. This lets us emit each element separately, for example:

    workflow {
        original_ch = Channel.from([
           ['a', [[1, 2, 3], [4, 5, 6]]],
           ['b', [[7, 8, 9], [10, 11, 12]]],
        ])
    
        transformed_ch = original_ch.flatten()
    
        transformed_ch.view()
    }
    

    Results:

    $ nextflow run main.nf 
    
     N E X T F L O W   ~  version 24.10.0
    
    Launching `main.nf` [berserk_noyce] DSL2 - revision: 7be21032be
    
    a
    1
    2
    3
    4
    5
    6
    b
    7
    8
    9
    10
    11
    12