javajsonspringstream

How to create a Java API to pass-through a stream of json lines from another source


I would like to create a Java Spring Boot project which offers a REST API to a client. A request to the API will be forwarded (with some changes) to a database. I would like to receive the response from the db and forward it to the client.

I know how to do this for a simple request where I get a single result from the database. But the db returns a stream of json lines and I would like to pass it through to the client directly, as a stream. So I do not want to collect the stream until it ends (storing it in memory) and then sent it to the client. I may also want to apply some conversion or filtering to every single json line. A buffer that stores a few lines as a cache would be ok.


Solution

  • If I understand correctly, you want to return one row at a time from a ResultSet - Spring Boot offers several mechanisms to do this, see - https://www.baeldung.com/spring-mvc-sse-streams

    A simple example, using StreamingResponseBody to write the database rows as a JSON array, might look something like:

    @Controller
    public class SomeController
    {
      @Autowired
      private final DataSource dataSource;
    
      @GetMapping
      public ResponseEntity<StreamingResponseBody> handleGet(@RequestParam("someParam") int someParam) 
      {
        return new ResponseEntity(query(someParam), HttpStatus.OK);
      }
    
      private StreamingResponseBody query(int someParam)
      {
        return out -> 
        {
          out.write((byte)'[');
    
          try (
            var connection = dataSource.getConnection();
            var preparedStatement = prepareStatement(connection, someParam);
            var resultSet = preparedStatement.executeQuery()
          )
          {
             while (resultSet.next())
             {
                out.write(resultSet.getBytes("some_column_containing_json")); 
                if (!resultSet.isLast()) out.write((byte)',');
             }
          }
          finally
          {
            out.write((byte)']');
          }
        };
      }
    
      private PreparedStatement prepareStatement(Connection connection, int someParam)
      {
         var preparedStatement = connection.prepareStatement("SELECT some_column_containing_json FROM some_table WHERE some_other_column = ?");
         preparedStatement.setInt(1, someParam);
    
         return preparedStatement;
      }
    }