javaparsingstreambytepipes-filters

Best way to parse data stream containing of pairs of id and data point


i am trying to write a simple pipes and filters system and I am currently parsing my input stream one byte at a time.

My data stream follows this format:

ID (4 Bytes) DATA (8 Bytes) ID (4 Bytes) DATA (8 Bytes)

Each data frame has 6 of those pairs of ids and data.

What is the best practice to parse such a stream? I want to be able to filter out the ids and some of the data, so that I will have a table of just DATA01, DATA02, etc.

How to I extract for example the first data point (long) after the first id (000, integer, for example) and then the second data point (double) after the second id (001, integer).

I hope I was able to elaborate my problem.

Thank you in advance and regards,

Philipp


Solution

  • ByteBuffer is a great class for accomplishing what you are trying to do. ByteBuffer allows you to directly convert byte arrays into their corresponding primitive values. This in combination with reading into a buffer instead of one byte at a time, you will have a relatively concise and efficient solution!

    Example:

        public void parseStream( InputStream is ) throws IOException 
        {
            boolean vtoggle = true; // Are we converting to long or double?
            ByteBuffer idBuffer = ByteBuffer.allocate( 4 ); // Initialize our id buffer
            ByteBuffer valueBuffer = ByteBuffer.allocate( 8 ); // Initialize our value buffer
    
            while( true /*or some real condition*/ )
            {
                idBuffer.put( readFromInput( is, 4 ) ); // Store the id bytes
                valueBuffer.put( readFromInput( is, 8 ) ); // Store the value bytes
                int id = idBuffer.getInt(); // Convert id bytes
                if( vtoggle )
                {
                    long lvalue = valueBuffer.getLong(); // Convert long bytes
                    // Do something with value
                }
                else
                {
                    double dvalue = valueBuffer.getDouble(); // Convert double bytes
                    // Do something with value
                }
                idBuffer.clear(); // Reset id buffer
                valueBuffer.clear(); // Reset value buffer
                vtoggle = !vtoggle; // Code to alternate whether or not we are converting to long or double
            }
        }
    
        /**
         * Read and return a certain number of bytes from our stream
         */
        public byte[] readFromInput( InputStream is, int count ) throws IOException
        {
            byte[] buffer = new byte[ count ];
            int bytesRead = 0;
            int offset = 0;
            while( ( bytesRead = is.read( buffer, offset, buffer.length - offset  ) ) > 0 )
            {
                offset += bytesRead;
            }
            if( offset == buffer.length )
            {
                return buffer;
            }
            else
            {
                throw new IOException( "Unexpected end to stream." );
            }
        }
    

    This is obviously just a template, but hopefully it guides to towards a proper solution to you problem.