Consumer quota management is used to set invocation quotas for consumers in AI Gateway, including Token usage quotas and cost budget quotas. Through quota management, you can control a consumer's model invocation volume, prevent resource abuse, and ensure business continuity.
Feature Overview
Consumer quota management supports the following core capabilities:
Token Usage Quota: This quota limits the total Token consumption (input tokens + output tokens) for a consumer within a specified time period.
Request Quantity Quota: This quota restricts the total number of requests a consumer can make within a specified time period.
Quota Period: The quota period can be set on a daily, weekly, or monthly basis.
Overrun Policy: You can configure a policy for handling quota overruns (rejecting requests / routing with degradation).
Quota Status: This section displays the quota usage status (Normal / Warning / Exceeded) for each consumer in real time.
Prerequisites
An AI gateway instance has been created, and its version is 3.9.4 or later.
A consumer (Consumer) has been created.
Operation Steps
Going to the Consumer Quota Management Page
2. On the instance list page, click the ID of the gateway instance you want to configure to go to its basic information page.
3. In the left sidebar, select Quota Management.
The quota configuration list and current quota usage for the gateway instance are displayed on the Quota Management page.
Adding a Consumer Quota Rule
You can select a consumer to add quota rules.
1. On the Quota Management page, click Create Quota Rule.
2. In the dialog box that appears, configure the following parameters:
|
Consumer | Yes | Select a consumer for the current instance. If you need to create a new one, click New Consumer. |
Quota type | Yes | Select Token usage, request count, or cost |
Statistical Cycle | Yes | Quota statistics period: Daily / Weekly / Monthly / Yearly |
Quota | Yes | Token Usage Quota: This quota limits the input Token usage (Unit: Token). Request Quantity Quota: Enter the maximum number of requests (Unit: requests). Cost: Enter the maximum usage cost (Unit: CNY). |
Over-limit Handling | Yes | Defaults to rejecting requests. Reject Requests: When the quota is exceeded, consumer requests are rejected. Allow Overrun: Consumers are still allowed to request model usage after the quota is exceeded. |
Enabling Status | Switch | Triggers an alert when quota utilization reaches a specified percentage. Valid range: 50% ~ 95%. |
3. Quota Type Description:
Quota limits can be tracked on a daily, weekly, or monthly basis.
When the quota is exceeded, the system rejects new requests.
You can configure multiple quota types (Token, Request Quantity) for the same object.
Quota usage is updated once per hour.
4. Click OK to complete the quota configuration addition.
Attention:
A consumer can configure multiple quotas simultaneously, such as a Token usage quota and a cost budget quota.
When multiple quotas are exceeded simultaneously, the overrun policy that is triggered first takes precedence.
The target model service for the fallback route must have been created and be in a normal state.
Viewing Quota Usage
After quota configuration is complete, the system tracks each consumer's quota usage in real time.
On the Quota Management page, the quota list displays the following information:
|
Resource ID/Name | Resource ID/Name corresponding to the quota rule |
Quota type | Token quantity quota / Request quantity quota |
Statistical Cycle | Daily / Weekly / Monthly |
Quota Limit | Configured quota limit value |
Usage | Quota amount used in the current period |
Utilization | Current usage / Quota limit × 100% |
Enabling Status | Current status of the quota rule |
Creation/Modification Time | Latest creation and modification time |
Operation | Edit / Delete |
Editing Consumer Quotas
You can modify existing quota configurations as needed.
1. On the Quota Management page, locate the target quota configuration.
2. Click Edit in the Actions column.
3. Modify the parameters that need to be changed (consumers cannot modify them).
4. Click OK to save the modifications.
Note:
After a quota is edited, the quota usage for the current period is not reset and continues to accumulate.
Deleting Consumer Quotas
When a quota is no longer needed, you can delete its configuration.
1. On the Quota Management page, locate the target quota configuration.
2. Click Delete in the Actions column.
3. In the confirmation dialog box, click OK.
Redis Configuration for Quota Management
Before using quota management, you must manually configure Redis to store current quota rule information. If Redis is not configured, quota data cannot be displayed.
1. On the Quota Management page, click Redis Configuration to go to the Redis Configuration page.
2. On the Redis Configuration page, click Edit Redis Configuration, fill in the Redis information, and complete the form based on your Redis service address and other details.
3. After clicking Save to complete the Redis configuration, you can start configuring quota rules.
Editing Redis Configuration
When using quota management, if you need to modify the Redis configuration, click the Edit Redis Configuration button to go to the Redis Edit page.
1. On the Quota Management page, click Edit Redis Configuration to go to the Redis Edit page.
2. On the Redis Edit page, modify the Redis service information.
3. After clicking Save to complete the Redis configuration modification, you can proceed to configure new quota rules.
Quota Rules
This quota limits the consumer's call volume for model services. By default, the quota for the Token quantity dimension is calculated based on the total number of tokens.
The gateway supports AI caching (exact match caching / semantic similarity caching). When a cache hit occurs, the gateway directly returns the request without querying the upstream model, so no model-side Tokens are consumed. You can use the "Gateway Cache Hit Request Statistics Method" configuration item to customize the statistical scope for Token quotas:
1. Count All (Default): Both cache hits and model calls are counted toward the quota.
2. Exclude Gateway Cache Hits: Only Tokens consumed by actual model calls are counted toward the quota, while cache hits are excluded, aligning the quota with actual model usage.
Directions
2. In the left sidebar, click Quota Management to go to the Quota Rules list page.
3. Click Create Quota Rule, or select an existing rule and click Edit to open the Quota Rule Configuration dialog.
4. In the dialog, locate Gateway Cache Hit Request Statistics Method:
Count All (Default): Tokens consumed by both cache hits and model calls are counted toward the quota.
Exclude Gateway Cache Hits: Only Tokens consumed by actual model calls are counted, while gateway cache hits are excluded.
5. Click Confirm to complete the configuration.
FAQs
Q1: How Is the Quota Cycle Calculated?
By Day: from 00:00:00 to 23:59:59 of the current day.
By Week: from 00:00:00 on Monday to 23:59:59 on Sunday.
By Month: from 00:00:00 on the 1st day of the month to 23:59:59 on the last day of the month.
Q2: Can a Consumer Be Configured with Multiple Quotas?
Yes. A consumer can configure multiple quotas simultaneously, such as a Token usage quota and a request quantity quota. They can also configure multiple quotas of the same type, for example, two Token usage quotas that are counted daily and monthly respectively.