Big data messaging with Kafka, Part 2 | JavaWorld

Big data messaging with Kafka, Part 2 | JavaWorld

More like this Email a friend From Sorry Partitions in Kafka Topics in Kafka can be subdivided into partitions. For example, while creating a topic named Demo, you might configure it to have three partitions. The server would create three log files, one for each of the demo partitions. When a producer published a message to the topic, it would assign a partition ID for that message. The server would then append the message to the log file for that partition only. If you then started two consumers, the server might assign partitions 1 and 2 to the first consumer, and partition 3 to the second consumer. Each consumer would read only from its assigned partitions. You can see the Demo topic configured for three partitions in Figure 1. Figure 1. A partitioned topic in Apache Kafka To expand the scenario, imagine a Kafka cluster with two brokers, housed in two machines. When you partitioned the demo topic, you would configure it to have two partitions and two replicas.

Read full article from Big data messaging with Kafka, Part 2 | JavaWorld