Im trying to use gpss
(Greenplum Stream Server) for loading data from Kafka to GreenplumDB.
Main question is how\when gpss
instance commit current writen offset to kafka?
Right now gpss
instance does not commit any message to kafka but handle current offset in service table in GreenplumDB.
Expected behavior for me is:
gpfdist
But right now its working without step n.5 Any one know why?
The second question is - does gpss use a group.id?
In gpss job config i found PROPERTIES
block config that correspond to kafka consumer config properties
gpss would commit consumed offset to Kafka if the 'group.id' is set in the yaml file since version 1.6.0. It only commited offset to the tracking table of Greenplum before.