Traffic Throttling Mechanisms
Last updated:2025-12-24 15:03:03
Cluster-Level Distributed Traffic Throttling
This traffic throttling method applies to the pro clusters of TDMQ for Apache Pulsar. When TDMQ for Apache Pulsar producers and consumers produce/consume massive messages at extremely high speeds, they consume server resources, causing saturation of CPU, memory, and network disk I/O resources. Therefore, TDMQ for Apache Pulsar designs a traffic throttling mechanism that sets different traffic throttling thresholds based on instance specifications to implement cluster self-protection. This prevents excessive resource consumption from impairing cluster quality and causing global stability risks.
Traffic Throttling Mechanisms
The producer traffic throttling mechanism of TDMQ for Apache Pulsar adopts the method of delayed response packet return, with a statistics window of 1 second.
Taking production TPS traffic throttling as an example:
If the production TPS is set to 100 and a user sends 100 messages within the first 400 ms of 1 second, the request for sending the 101st message will need to wait 600 ms before being processed.
From the producer's perspective, when producer traffic throttling occurs, the time consumed for sending messages will increase, and message sending timeouts may even occur.
From the consumer's perspective, when consumption traffic throttling occurs, the end-to-end latency of messages from production to consumption will increase, and a message backlog may occur.
Description of Traffic Throttling Principles
Producer:
The traffic throttling statistics window is 1 second. When the quota within the statistics window is exhausted, the server will close all producer channels and stop accepting message sending requests. The producer channels will reopen at the next time window to resume processing message sending requests.
Consumer:
The traffic throttling statistics window is 1 second. When the quota within the statistics window is exhausted, the server stops pushing messages to consumers until the next time window.
Note:
How to understand the closure of channels after producer traffic throttling?
When producer traffic throttling occurs, the server closes the TCP connection channel corresponding to the producer. After the channel is closed, the server will no longer accept requests from this TCP connection until the TCP connection channel is reopened.
Use Cases of TDMQ for Apache Pulsar Distributed Traffic Throttling
1. We recommend that users select cluster specifications based on the actual peak production/consumption volumes of their business. Set the production/consumption allocation ratio for traffic throttling according to the fan-out ratio of production and consumption. We recommend that users perform stress tests before the official launch to assess whether the cluster capacity meets requirements.
2. For non-delayed messages, do not set delayed message fields. If the sender sets a delayed message field, the server will treat the message as delayed and include it in delayed message statistics rate, regardless of the delay duration. A typical scenario: Taking Java as an example (the same applies to other SDKs such as Go), a message will be considered a delayed message as long as
deliverAfter or deliverAt is set during message sending, even if the value is 0 or earlier than the current time.3. Configure alarms for the production/consumption rate and bandwidth of the cluster. When the production/consumption rate and bandwidth of the cluster exceed 80% of the specified specifications, it is recommended to upgrade the Pro Edition instance specifications to avoid the risk of increased latency due to traffic throttling.
4. Configure alarms for the number of production/consumption traffic throttling events. When the alarm is triggered, it indicates that production/consumption has exceeded limits within a second-level window. It is recommended to promptly upgrade the Pro Edition instance specifications to avoid the risk of increased latency due to traffic throttling.
FAQs
Question 1: Why is traffic throttling triggered when production/consumption volumes are lower than specifications?
As described in the traffic throttling principles above, traffic throttling is measured in seconds, and the monitoring data in the console is collected and reported in minutes. The calculation formula for production/consumption statistics on the monitoring platform is [message volume within 1 min/60]. When the production/consumption volumes of the client are unevenly distributed within 1 minute, the volumes may be highly concentrated in 1 second or a few seconds of the 1-minute window, thus exceeding the quota specified in the traffic throttling window, while remaining much lower than the quota during other periods. In this case, the production/consumption volumes displayed on the monitor platform are lower than the instance specifications, but traffic throttling is still triggered.
Question 2: Why does the production/consumption peak exceed the instance specifications?
Scenario 1: TDMQ for Apache Pulsar is a distributed system where a TDMQ for Apache Pulsar node consists of multiple broker nodes. At the same time (within a single throttling window), traffic throttling is implemented by each node, and the traffic throttling threshold of each node is the remaining threshold of the current cluster. For example, if the traffic throttling threshold of a cluster is 1000 and there are 5 broker nodes, when actual usage reaches 750 (assuming the usage of each node is even at 150), the traffic throttling threshold of each node at this time is 400 (150 + 1000 - 750). In this case, the instantaneous traffic may actually reach 2000 (400 x 5), resulting in a situation where the traffic exceeds the specification within a traffic throttling window.
Scenario 2: As described in the traffic throttling principles above, when traffic throttling occurs, the write channel is closed. However, current requests (even those exceeding the traffic throttling threshold) will continue to be processed. Therefore, when the number of concurrent requests is large, the volume in a single statistical window may exceed the traffic throttling threshold.
Question 3: How do I determine whether traffic throttling occurs in TDMQ for Apache Pulsar?
Check the cluster monitoring information on the monitoring page of the TDMQ for Apache Pulsar Pro Edition console. If the number of traffic throttling events is greater than 0, it means traffic throttling has occurred.
Topic Partition Traffic Throttling
This traffic throttling method applies to all types of TDMQ for Apache Pulsar clusters.
Traffic Throttling Principle Description
Producers
Description of server traffic throttling logic: Producer throttling is imprecise and relies on internal scheduled tasks (executed once every 50 ms by default) to check whether the production volume of each partition exceeds the quota within a 1-second window.
Behavior of the server after traffic throttling: The producer uses the soft traffic throttling method. When traffic throttling occurs, the server closes the read channel of the producer corresponding to the topic and stops processing production requests. It will resume the producer's read channel after waiting for up to 1 second, allowing the resumption of message sending requests until traffic throttling is triggered again.
Client behavior after traffic throttling: When traffic throttling occurs, the time consumed for message sending will increase, and message sending timeouts may occur.
Consumers
Description of server traffic throttling logic: Consumer traffic throttling is imprecise. It checks whether the consumption TPS and bandwidth within a 1-second time window exceed the quota.
Behavior of the server after traffic throttling: The server stops pushing messages to consumers for 1 second.
Client behavior after traffic throttling: When traffic throttling occurs, the end-to-end latency of messages from producer to consumer will increase, and a message backlog may occur.
Use Cases of TDMQ for Apache Pulsar Topic Partition Traffic Throttling
1. Single-topic partitions have limits on production and consumption TPS and bandwidth. If the TPS and bandwidth concurrency of the topic are relatively high, it is necessary to expand partitions.
2. Configure alarms for the production and consumption rate of the topic and the percentage of traffic quota used. When the usage exceeds 80%, it is recommended to increase the number of partitions to avoid triggering traffic throttling on single-topic partitions.
FAQs
Question 1: Why can the production/consumption traffic of a partition exceed the traffic throttling threshold?
As described in the traffic throttling principles above, topic partitions adopt an imprecise soft-limit traffic throttling algorithm. Based on the throttling logic of the producer and consumer, both production and consumption may experience traffic exceeding the traffic throttling threshold.
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No
Feedback