The statuses of a consumer group on the consumer group list page include Dead, Empty, PreparingRebalance, AwaitingSync, and Stable, with the most common being Empty, Stable, and Dead. Status mechanism transitions in the consumer group are shown below:
Dead: No members exist in the consumer group and no metadata is present.
Empty: No members exist in the consumer group, but metadata is present. If all offsets in the group have expired, the consumer group will become Dead. Generally, a newly created group is in the Empty status by default.
In the Apache Kafka version 0.10.x, when there are no members in a consumer group and this status persists for more than 7 days, the consumer group will be automatically deleted. TDMQ for CKafka (CKafka) supports customizing a retention period for empty consumer groups in the instance dimension. For detailed information, see Configuring a Retention Period for Empty Consumer Groups.
Stable: Members in the consumer group can consume normally.
Rebalance Status Details
Cause of Rebalance
According to the status mechanism of a consumer group, when the consumer group is in the Empty, AwaitSync, or Stable status, it may perform a rebalance. Rebalances may occur in the following scenarios:
A consumer has subscribed to a topic.
A consumer is disabled.
When a consumer is considered to be in the Dead status by the group coordinator.
If a consumer fails to send a heartbeat to the group coordinator within session.timeout.ms, the consumer will be considered in the Dead status, triggering a rebalance. For details, see Configuring Client Parameters.
The number of partitions has increased.
A non-existent topic has been subscribed to.
If you have subscribed to a topic that has not been created, a rebalance will occur when this topic is created. Similarly, a rebalance will also occur if a subscribed topic is deleted.
The application crashes.
Analysis of the Rebalance Process
Taking the mechanism of Kafka version 0.10 as an example, the rebalance process is analyzed as follows:
1. When any consumer wants to join a consumer group, a JoinGroup request will be sent to the group coordinator. The first consumer to join the group will become the group leader.
2. The leader will receive the list of all consumers in this group from the group coordinator and is responsible for assigning partitions to consumers in the group. Partition assignment can be implemented through the PartitionAssignor API.
3. After the assignment is completed, the leader sends the assignment results to the group coordinator, who then distributes the results to all consumers.
4. Therefore, each consumer can only view its assigned partition, and the leader is the only consumer who can access consumers in the consumer group and the nodes of their respective partition assignments.
The above process will be executed once whenever a rebalance occurs.
