filtertuplesnextflow

Nextflow filter entire tuple based on one value

Needing some extra guidance on filtering an entire tuple based on one value.

I have a channel of tuples (reads_ch):

[[id:S1, single_end:false], [/home/ubuntu/S1.R1.fastq.gz, /home/ubuntu/S1.R2.fastq.gz], /home/ubuntu/S1.txt, [PASSED: File S1_R1 is not corrupt., PASSED: File S1_R2 is not corrupt.]]
[[id:S2, single_end:false], [/home/ubuntu/S2.fastq.gz, /home/ubuntu/S2.R2.fastq.gz], /home/ubuntu/S2.txt, [PASSED: File S2_R1 is not corrupt., PASSED: File S2_R2 is not corrupt.]]
[[id:S3, single_end:false], [/home/ubuntu/S3.R1.fastq.gz, /home/ubuntu/S3.R2.fastq.gz], /home/ubuntu/S3.txt, [FAILED, FAILED]]

I am trying to filter this for any sample that did not 'fail'. I've gotten as far as clearing it[3] but haven't figured out how to remove that sample. while keeping the structure of the remaining samples (S1,S2)

Goal:

[[id:S1, single_end:false], [/home/ubuntu/S1.R1.fastq.gz, /home/ubuntu/S1.R2.fastq.gz], /home/ubuntu/S1.txt, [PASSED: File S1_R1 is not corrupt., PASSED: File S1_R2 is not corrupt.]]
[[id:S2, single_end:false], [/home/ubuntu/S2.fastq.gz, /home/ubuntu/S2.R2.fastq.gz], /home/ubuntu/S2.txt, [PASSED: File S2_R1 is not corrupt., PASSED: File S2_R2 is not corrupt.]]

Attempt:

passing = read_ch.map { meta, reads, outcome_file, outcome_status ->
tuple(meta, reads, outcome_file, outcome_status.findAll { !it.contains('FAILED') })
}
passing.view()

[[id:S1, single_end:false], [/home/ubuntu/S1.R1.fastq.gz, /home/ubuntu/S1.R2.fastq.gz], /home/ubuntu/S1.txt, [PASSED: File S1_R1 is not corrupt., PASSED: File S1_R2 is not corrupt.]]
[[id:S2, single_end:false], [/home/ubuntu/S2.fastq.gz, /home/ubuntu/S2.R2.fastq.gz], /home/ubuntu/S2.txt, [PASSED: File S2_R1 is not corrupt., PASSED: File S2_R2 is not corrupt.]]
[[id:S3, single_end:false], [/home/ubuntu/S3.R1.fastq.gz, /home/ubuntu/S3.R2.fastq.gz], /home/ubuntu/S3.txt, []]
Solution

  • The entire channel element is supposed to be removed even if only one sample failed? If that's what you want, you'll find the solution below:

    Preparing a fake channel similar to yours:

    Channel
  .of([
        [id:'S1', single_end:false],
        [file('/home/ubuntu/S1.R1.fastq.gz'), file('/home/ubuntu/S1.R2.fastq.gz')],
        file('/home/ubuntu/S1.txt'),
        ['PASSED: File S1_R1 is not corrupt.', 'PASSED: File S1_R2 is not corrupt.']
      ],
      [
        [id:'S2', single_end:false],
        [file('/home/ubuntu/S2.fastq.gz'), file('/home/ubuntu/S2.R2.fastq.gz')],
        file('/home/ubuntu/S2.txt'),
        ['PASSED: File S2_R1 is not corrupt.', 'PASSED: File S2_R2 is not corrupt.']
      ],
      [
        [id:'S3', single_end:false],
        [file('/home/ubuntu/S3.R1.fastq.gz'), file('/home/ubuntu/S3.R2.fastq.gz')],
        file('/home/ubuntu/S3.txt'),
        ['FAILED', 'FAILED']
      ])
  .set { my_ch }

    Filtering:

    my_ch
  .filter { it[3].findAll { !it.contains('FAILED') } }
  .view()

    Output:

    enter image description here

    You can also do it like this:

    my_ch
  .filter { it[3][0] != 'FAILED' && it[3][1] != 'FAILED' }
  .view()

    which makes it easy to adapt if you want either of one failing, instead of both.