Scenarios
This document describes how to configure the logging format for request/response bodies in AI Gateway logs. Using this feature, you can choose to write the Prompt/Completion to the LLM Log in either the OpenAI-compatible format (normalized across vendors) or as the raw body (the actual request body forwarded to the vendor). This meets the requirements for two different scenarios: log analysis and problem debugging.
In highly regulated scenarios such as finance and government, you can also enable "No truncation (retain full context)" to record the complete request/response Body regardless of the truncation length, if you need to retain complete LLM request/response records for audit and traceability.
Prerequisites
Enabling Package Collection
2. On the instance list page, click the ID of the gateway instance you want to configure to go to its basic information page.
3. Choose Model Management > Model API.
4. Locate the target model API and click the API name to go to its details page.
5. Go to the Basic Information tab. In the Log Configuration section:
Enable the Request Body Collection (Prompt) switch as needed. The gateway will record the input parameter messages based on the configured body truncation length.
Enable the Response Body Collection (Completion) switch as needed. The gateway will record the model-returned content based on the configured body truncation length.
After body collection is enabled, the messages content from requests is written to the LLM Log. Please confirm that the content does not contain user privacy data and complies with corporate compliance requirements. If desensitization is required, you can combine log desensitization rules with data security configurations.
Configuring Non-Truncation (Preserving Full Context)
By default, the gateway truncates the body based on the "Content Truncation Length". When you need to retain the full context for compliance auditing, follow these steps to enable no truncation:
1. In the Log Configuration section above, locate the Content Truncation Length configuration item.
2. Select the No Truncation (Retain Full Context) switch below it. After selection, the complete Body is recorded regardless of the truncation length.
3. After you select the switch, a risk reminder pop-up is forcibly displayed. You must complete the following three compliance self-checks and select all of them before the Confirm Enable button becomes clickable:
The user request content of this known API will be recorded.
Log storage and access permissions comply with data compliance requirements.
No highly sensitive personal information (such as ID numbers or passwords) is recorded.
Important trade-offs after "No Truncation" is enabled:
All request and response text will be written to log storage, and the estimated log storage volume is expected to increase by 3 to 5 times.
You must configure sufficient quota for CLS delivery.
Confirm that log content does not contain users' personal privacy data, log storage complies with corporate data retention requirements, and log access permissions are properly controlled.
Parameter Description
|
Request body capture (Prompt) | switch | No | Disabled. | Record request body content after this feature is enabled. |
Response body capture (Completion) | switch | No | Disabled. | Record response body content after this feature is enabled. |
Content truncation length | number | No | 32768 | Unit: byte; takes effect when "Do not truncate" is not selected. |
Do not truncate (retain full context) | switch | No | Disabled. | After this option is selected, the truncation length is ignored and the complete Body is recorded. When this feature is enabled, a compliance self-check dialog box is forcibly displayed. |