Custom Partitioner for Kafka Producer using application.properties | Absolute beginner guide
Kafka Producer with custom partitioner Configuration for confluent cloud
Table of contents
I almost gave up trying to configure Custom Partitioner Class using application.properties.
spring.kafka.producer.partitioner.class=classPath
Does not work after trying a hell lot of things. The partitioner.class remained null everytime.
I even tried to configure Custom Partitioner with code as well, but as my Kafka was on confluent cluster, I needed to provide sasl and jaas config via code, and I was not able to find any help online for that as well.
The correct configuration for it is:
spring.kafka.properties.partitioner.class=com.puneetchhabra.NotificationProcessor.config.CustomPartitioner
//Provide your complete classpath
Creating CustomPartitioner class
public class CustomPartitioner implements Partitioner {
@Override
public int partition(String topic, Object key, byte[] keyBytes, Object value, byte[] valueBytes, Cluster cluster) {
//Your custom logic for partition
//return partition index
return 0; //every message sent to partition-0
}
@Override
public void close() {
// Perform any necessary cleanup
}
@Override
public void configure(Map<String, ?> configs) {
// Perform any necessary configuration
}
}