Don't Use Apache Kafka Consumer Groups the Wrong Way! - DZone Big Data

Don't Use Apache Kafka Consumer Groups the Wrong Way! - DZone Big Data

Having consumers as part of the same consumer group means providing the "competing consumers" pattern with whom the messages from topic partitions are spread across the members of the group. Each consumer receives messages from one or more partitions ("automatically" assigned to it) and the same messages won't be received by the other consumers (assigned to different partitions). In this way, we can scale the number of the consumers up to the number of the partitions (having one consumer reading only one partition); in this case, a new consumer joining the group will be in an idle state without being assigned to any partition.

Read full article from Don't Use Apache Kafka Consumer Groups the Wrong Way! - DZone Big Data