Intent Recognition Routing
Download
Focus Mode
Font Size
Scenarios
An intent recognition routing policy automatically identifies the intent category of a user request based on its semantic content and routes the request to the corresponding model service. It is applicable in the following scenarios:
Multi-Task Model Distribution: It routes different tasks, such as translation, code generation, and conversation, to the optimal model based on user intent.
Cost Optimization: Simple intents, such as casual conversation, are routed to low-cost models, while complex intents, such as code review, are routed to high-performance models.
Tiered Service: VIP user intents are routed to dedicated services, while regular users are routed to shared services.
Scenario Isolation: It isolates requests from different business scenarios into separate model services to prevent mutual interference.
Note:
Intent recognition routing requires selecting an intent recognition model service to perform semantic analysis. This model is used to understand the intent of user requests and does not directly process user business requests.
Prerequisites
The AI gateway instance has been created and is in a running state.
At least two model services (an intent recognition model + a business model) have been created.
The intent categories to be identified and their corresponding routing targets have been defined.
Operation Steps
Step 1: Configure the Model API
1. Log in to the Microservices Platform console. In the left sidebar, click AI Gateway > Instance List.
2. On the instance list page, click the ID of the gateway instance you want to configure to go to its basic information page.
3. In the left sidebar, choose Model Management > Model API.
4. Click New or edit an existing API.
5. After completing the basic information configuration, go to Step 2: Select Model Service.
6. Select Service Type as Multi-Model Service.
7. In the Routing Policy area, select Intent Recognition Routing.
Step 2: Configure the Intent Recognition Model
In the Intent Recognition Model Configuration area, select the model for intent recognition.
Configuration Parameter Description:
Parameter | Required | Description |
Intent Recognition Model | Yes | Model service for intent recognition |
Confidence Threshold | Yes | Use the default service when the intent recognition confidence score is below this threshold. |
Default Model Service | Yes | Service used when no intent is matched or the confidence score is insufficient. |
Recommended Confidence Threshold:
Threshold | Description | Applicable Scenarios |
0.50 | Route more requests based on the recognized intent with a lower confidence requirement | When the number of intent categories is small and they are highly distinguishable |
0.70 | Recommended confidence threshold that balances accuracy and coverage | Most scenarios |
0.85 | Route intents by category only when their confidence is high, reflecting a higher confidence requirement. | Scenarios requiring high routing accuracy |
Step 3: Configure the Intent Type and Model Mapping
In the Routing Rule Configuration area, configure the model service for each intent to support scenarios such as code writing, mathematical calculation, translation, quick Q&A, and complex reasoning.
Step 4: Save the Configuration
Click OK to save the Model API configuration. The intent recognition route takes effect immediately.
Must-Knows
Intent Recognition Model Selection: The intent recognition model must have strong natural language understanding (NLU) capabilities.
Additional Token Consumption: Each request is first routed to the intent recognition model for semantic analysis before routing, resulting in additional Token consumption and time overhead.
Intent Requests Not Logged: The requests and responses from intent recognition calls are not written to the LLM Log (serving only for routing decisions).
Confidence Threshold Tuning: It is recommended to first use a lower threshold (0.5) to observe the recognition performance, and then adjust it gradually based on the actual performance.
Increased Routing Latency: Routing latency may increase because the intent recognition model must be called first (depending on the model's response speed).
Help and Support
Was this page helpful?
You can also Contact sales or Submit a Ticket for help.
Help us improve! Rate your documentation experience in 5 mins.
Feedback