javascalaioinputstream

How to chain multiple different InputStreams into one InputStream

I'm wondering if there is any idiomatic way to chain multiple InputStreams into one continual InputStream in Java (or Scala).

What I need it for is to parse flat files that I load over the network from an FTP- server. What I want to do is take files [1..N], open up streams, and then combine them into one stream. So when file 1 comes to an end, I want to start reading from file 2, and so on, until I reach the end of file N. 

I need to read these files in a specific order. Data comes from a legacy system that produces files in batches, so data in one file depends on data in another file, but I would like to handle them as one continuous stream to simplify my domain logic interface. 

I searched around and found PipedInputStream, but I'm not positive that is what I need. An example would be helpful.

Solution

  • It's right there in JDK! Quoting JavaDoc of SequenceInputStream:

    A SequenceInputStream represents the logical concatenation of other input streams. It starts out with an ordered collection of input streams and reads from the first one until end of file is reached, whereupon it reads from the second one, and so on, until end of file is reached on the last of the contained input streams.

    You want to concatenate arbitrary number of InputStreams while SequenceInputStream accepts only two. But since SequenceInputStream is also an InputStream you can apply it recursively (nest them):

    new SequenceInputStream(
    new SequenceInputStream(
        new SequenceInputStream(file1, file2),
        file3
    ),
    file4
);

    ...you get the idea.

    See also