I'm trying to implement a Kafka producer/consumer model, and am deliberating whether creating a separate publisher thread per topic would be preferred over having a single publisher handle multiple topics.
By separate publisher thread, I think you mean separate producer objects. If so..
Since messages are stored as key-value pairs in Kafka, different topics can have different key-value types. So if your Kafka topics have different key-value types like for example..
Topic1 - key:String, value:Student
Topic2 - key:Long, value:Teacher
and so on, then you should be using multiple producers. This is because the KafkaProducer class at the time of constructing the object asks you for the key and value serializers.
Properties props=new Properties();
props.put("key.serializer",StringSerializer.class);
props.put("value.serializer",LongSerializer.class);
KafkaProducer<String,Long> producer=new KafkaProducer<>(props);
Though, you may also write a generic serializer for all the types! But, it is better to know before hand what we are doing with the producer.