JRecord - Formatting file transferred from Mainframe

I am trying to display a mainframe file in a eclipse RCP application using JRecord library.I already have the COBOL copybook as a text file. to accomplish that,

1. I am transferring the file from mainframe to my desktop through apache commons net FTPClient API
2. Now I have a text file
3. I am removing the newline and carriage return characters
4. then I read it via ., a CobolIoProvider and convert it into a ArrayList of type AbstractLine

But I have offset issues because of some special charcters . here are the issues

1. when I dont perform step #3 , there are offset issues right from record 1. hence I included step #3
2. even when I perform step #3 , the first few thounsands of records seem to be formatted(or read ) by the AbstractLineReader correctly unless it encounters a special character (not sure but thats my assumption).

Code snippet:

ArrayList<AbstractLine> lines = new ArrayList<AbstractLine>();
        InputStream copyStream;
        InputStream fis;
        try {
            copyStream = new FileInputStream(new File(copybookfile));

            String filec = FileUtils.readFileToString(new File(datafile));
            System.out.println("initial len: "+filec.length());
            filec=filec.replaceAll("\r", "");
            filec=filec.replaceAll("\n", "");
            System.out.println("initial len: "+filec.length());

            fis= new ByteArrayInputStream(filec.getBytes());

            CobolIoProvider ioProvider = CobolIoProvider.getInstance();
            AbstractLineReader reader = ioProvider.newIOBuilder(copyStream, "REQUEST",
                    Convert.FMT_MAINFRAME).newReader(fis);
            AbstractLine line;
            while ((line = reader.read()) != null) {
                lines.add(line);
            }
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

What am I missing here ? is there an additional preprocessing that I need to do for the file transferred from mainframe ?

If it is a Text File (no binary data) with \r\n line delimiters try:

    ArrayList<AbstractLine> lines = new ArrayList<AbstractLine>();
    InputStream copyStream;
    InputStream fis;
    try {
        copyStream = new FileInputStream(new File(copybookfile));

        AbstractLineReader reader = CobolIoProvider.getInstance() 
            .newIOBuilder(copyStream, "REQUEST", ICopybookDialects.FMT_MAINFRAME)
                .setFileOrganization(Constants.IO_STANDARD_TEXT_FILE)
            .newReader(datafile);
        AbstractLine line;
        while ((line = reader.read()) != null) {
            lines.add(line);
        }
    } catch (FileNotFoundException e) {
        e.printStackTrace();
    } catch (IOException e) {
        e.printStackTrace();
    }

Note: The setFileOrganization tells JRecord what type of file it is. So .setFileOrganization(Constants.IO_STANDARD_TEXT_FILE) tells JRecord it is a Text file with \n or \r\n end-of-line markers. Here is a Description of FileOrganisation in JRecord.

The special charcters worry me though, if there is a \n in the 'Data' it will be treated as an end-of-line. You may need to do binary transfer and keep the RDW (Record-Descriptor-Word) if it is a VB file.

If The file contains Binary data, you will need:

• do a binary transfer (with RDW if it is a VB file)
• use the appropriate File-Organisation
• Specify Ebcdic (.setFont("cp037") tells JRecord is US-Ebcdic)

---

I will add a second answer for Generating Code using the RecordEditor

---

If you are absolutely sure all the records are the same length you can use the low-level routines to do the reading see the ReadAqtrans.java program in https://sourceforge.net/p/jrecord/discussion/678634/thread/4b00fed4/

basically you would do:

    ICobolIOBuilder iobuilder = CobolIoProvider.getInstance()
            .newIOBuilder("copybookFileName", ICopybookDialects.FMT_MAINFRAME)
                    .setFont("CP037")
                    .setFileOrganization(Constants.IO_FIXED_LENGTH);
    LayoutDetail layout = iobuilder.getLayout();
    FixedLengthByteReader br 
         = new FixedLengthByteReader(layout.getMaximumRecordLength() + 2);
    br.open("...");

    byte[] bytes;
    while ((bytes = br.read()) != null) {           
        lines.add(iobuilder.newLine(bytes));
    }