amazon-web-servicesamazon-kinesis

Are Kinesis sequence numbers unique across the stream or just per shard and per partition key?


The documentation is conflicting in this regard.

The PutRecords documentation says:

The SequenceNumber parameter is an identifier assigned to the put record, unique to all records in the stream.

while the Kinesis terminology and concepts documentation says:

Each data record has a sequence number that is unique per partition-key within its shard.

which is a much looser condition.

Which of these is right?


Solution

  • The SequenceNumber in Kinesis streams is only guaranteed to be unique within each shard (partition key is what determines the shard).

    Another reference of this can be found on the following awslabs GitHub issue: https://github.com/awslabs/amazon-kinesis-client/issues/643

    However, if hash value for the two clients using the same partition key are different causing the record to go to different shard, its possible they get the same sequence number. But the records would belong to different shards in this scenario.