Log Shipping
Last updated: 2025-12-31 16:41:45
Database audit of TencentDB for MySQL provides the log shipping feature. Through log shipping, database audit logs of TencentDB for MySQL can be collected and shipped to Cloud Log Service (CLS) for aggregation, management, and analysis. Logs can also be shipped to TDMQ for CKafka (CKafka). After shipping, real-time stream computing can be performed on the logs in the CKafka console. In addition, logs can be shipped to Cloud Object Storage (COS) for ARCHIVE storage. This document describes how to configure the log shipping feature of database audit via the console.
Prerequisites
Prerequisites for shipping logs to CLS:
Before using this feature, make sure you have activated CLS.
The instance is running.
Prerequisites for shipping logs to CKafka:
A routing policy has been added for CKafka instances.
The instance is running.
Prerequisites for shipping logs to COS:
Before using this feature, make sure you have activated COS.
The instance status is Running.
Supported Versions and Architectures
MySQL 5.6 20180101 and later versions.
MySQL 5.7 20190429 and later versions.
MySQL 8.0 20210330 and later versions.
The instance architecture is two-node, three-node, or Cluster Edition.
Log Shipping Billing
The feature of shipping TencentDB for MySQL database audit logs to CLS involves the third-party independently billed cloud service CLS. For billing details, see Cloud Log Service > Billing Overview.
The feature of shipping TencentDB for MySQL database audit logs to CKafka involves the third-party independently billed cloud service CKafka. For billing details, see TDMQ for CKafka > Billing Overview.
The feature of shipping TencentDB for MySQL database audit logs to COS involves COS, which is an independently billed third-party cloud product. For pricing, see Billing Overview.
After the log shipping feature is enabled for database audit of TencentDB for MySQL, traffic fees will be incurred. The fees are charged based on the traffic of shipped logs. For details, see the table below.
Note:
After the log shipping feature is enabled, traffic fees are incurred. However, regardless of whether you enable one or more log shipping paths (CLS, CKafka, or COS), the system only charges traffic fees incurred by this feature as a whole.
Billing Item: Audit Log Traffic | |
Chinese Mainland (USD/GB) | Hong Kong (China) and Other Countries and Regions (USD/GB) |
0.05882353 | 0.08823529 |
Log Shipping Traffic Monitoring
After the log shipping feature is enabled, you can use the monitoring feature to learn about the real-time shipping traffic generated by log shipping.
Monitoring Metric Name | Callable Metric Name | Unit | Metric Description |
Shipping traffic | AuditDeliverRate | MB | Shipping traffic generated by the log shipping feature. |
You can find an instance with the log shipping feature enabled in the audit instance list, click the monitoring icon in the Log Shipping column, and view the shipping traffic monitoring data.
Log Shipping Status Display
As shown above, the audit log shipping status of instances is displayed in the Log Shipping column on the Database Audit page in the TencentDB for MySQL console. The descriptions of each shipping status are as follows.
CKafka: Indicate that you have enabled shipping database audit logs of an instance to CKafka.
CLS: Indicate that you have enabled shipping database audit logs of an instance to CLS.
Disabled: Indicate that you have not enabled shipping database audit logs of an instance.
COS: Indicate that you have enabled shipping database audit logs of an instance to COS.
Related Documentation
For the operation steps of shipping database audit logs to CLS, CKafka, and COS, see the guides on the following tabs.
Enabling Log Shipping to CLS
1. Log in to the TencentDB for MySQL console.
2. Click Database Audit in the left sidebar.
3. Select a region at the top, go to the Audit Instance page, click Audit Storage Status, and select the Enabled option to filter instances with database audit enabled.
4. Find the target instance in the audit instance list (or filter by resource attributes in the search box) and choose More > Configure Log Shipping in the Operation column.
5. (Skip this step if CLS has already been activated.) Click go to activate in the pop-up sidebar to activate CLS.
6. (Skip this step if CLS has already been activated.) Return to the console after activation and click Activation Completed in the pop-up window for activation confirmation.
Note:
During the activation process, the system will verify whether activation is successful. If the system prompts that activation has failed, wait for a while and try again.
7. (Skip this step if CLS has already been authorized.) Click Go to Authorize in the sidebar and click Grant in the Service Authorization pop-up window.
Note:
During the authorization process, the system will verify whether authorization is successful. If the system prompts that authorization has failed, wait for a while and try again.
8. Click Enable now in the Ship to CLS area in the sidebar.
9. Complete the following configurations in the pop-up window and click Enable now.
Parameter | Description |
Destination region | Select the region for log shipping. If CLS supports the region of the database instance, the instance region will be selected by default. You can also select other available regions. If CLS does not support the region of the database instance, you can select another region supported by CLS. |
Log topic operations | Available options: Select existing log topic and Create log topic. |
Select existing log topic | If you choose the Select existing log topic option, you need to select an existing logset and log topic. Logset: Logsets classify log topics to facilitate log topic management. You can filter existing logsets in the search box. Log topic: A log topic is the basic unit for collecting, storing, retrieving, and analyzing log data. You can filter log topics of the selected logset in the search box. Note： Log topics that can be selected in this step should be those created with the Create Log Topic option selected for log topic operations when enabling log shipping in the console. Log topics created in the CLS console cannot be selected. |
Create Log Topic | If you choose the Create log topic option, you need to create a log topic and add it to an existing logset or a newly created logset. Log topic: A log topic is the basic unit for collecting, storing, retrieving, and analyzing log data. You need to create a log topic. Select the existing logset: The log topic to be created will be added to an existing logset. If you select this option, you can filter existing logsets in the search box. Create logset: The log topic to be created will be added to a newly created logset. If you select this option, you need to create a logset. |
Viewing Information About Log Shipping to CLS
After the feature of shipping database audit logs to CLS is enabled for an instance, you can view the relevant information on log shipping to CLS (view the logset and log topic for log shipping).
1. Log in to the TencentDB for MySQL console.
2. Click Database Audit in the left sidebar.
3. Select a region at the top, find the target instance on the Audit Instance page (or filter by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
4. View the log shipping information in the pop-up sidebar.
5. Click the logset name, log topic name, or Search and Analysis button to view relevant log shipping information in the CLS console.
Disabling Log Shipping to CLS
Note:
After log shipping is disabled, database audit logs of an instance will no longer be shipped. Note: After log shipping is disabled, only the shipping of new logs is stopped. The existing logs are still stored in the log topics until they expire, and storage fees will continue to be incurred during this period. If you need to delete the log topics, go to Log Topic Management.
1. Log in to the TencentDB for MySQL console.
2. Click Database Audit in the left sidebar.
3. Select a region at the top, find the target instance on the Audit Instance page (or filter by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
4. Click Disable Shipping in the upper right corner of the Ship to CLS area in the pop-up sidebar.
5. Read the precautions in the pop-up window, select Disable, and click OK.
Enabling Log Shipping to CKafka
1. Log in to the TencentDB for MySQL console.
2. Click Database Audit in the left sidebar.
3. Select a region at the top, go to the Audit Instance page, click Audit Storage Status, and select the Enabled option to filter instances with database audit enabled.
4. Find the target instance in the audit instance list (or filter by resource attributes in the search box) and choose More > Configure Log Shipping in the Operation column.
5. (Skip this step if CKafka has already been activated.) Click go to activate in the pop-up sidebar to activate CKafka.
6. (Skip this step if CLS has already been activated.) Return to the console after activation and click Activation Completed in the pop-up window for activation confirmation.
Note:
During the activation process, the system will verify whether activation is successful. If the system prompts that activation has failed, wait for a while and try again.
7. (Skip this step if CLS has already been authorized.) Click Go to Authorize in the sidebar and click Grant in the Service Authorization pop-up window.
Note:
During the authorization process, the system will verify whether authorization is successful. If the system prompts that authorization has failed, wait for a while and try again.
8. Click Enable Immediately in the Ship to TDMQ for CKafka area in the pop-up sidebar.
9. Complete the following configurations in the pop-up window and click OK.
Parameter | Description |
Target Region | Select the region for log shipping. If CKafka supports the region of the database instance, the instance region will be selected by default. You can also select other available regions. If CKafka does not support the region of the database instance, you can select another region supported by CKafka. |
CKafka Instance | Select a CKafka instance in the target region. Note: Note: Audit log shipping is supported only in CKafka 2.4.1 and later versions. CKafka instances of other versions do not support it. |
Topic | Select a topic for shipping. If no topic is available, you can create one. For operations, see Creating Topic. |
Viewing Information About Log Shipping to CKafka
After the feature of shipping database audit logs to CKafka is enabled for an instance, you can view the relevant information on log shipping to CKafka (view the CKafka instance ID, CKafka topic ID/name, region, and creation time for log shipping).
1. Log in to the TencentDB for MySQL console.
2. Click Database Audit in the left sidebar.
3. Select a region at the top, find the target instance on the Audit Instance page (or filter by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
4. View the log shipping information in the pop-up sidebar.
5. Click the CKafka instance ID, CKafka topic ID/name, and Message Query button to view instance details and query messages in the CKafka console.
Modifying Shipping Settings
After database audit log shipping to CKafka is enabled, you change the CKafka instance, region, or topic (CKafka topic ID/name) if needed. For details, see the steps below.
1. Log in to the TencentDB for MySQL console.
2. Click Database Audit in the left sidebar.
3. Select a region at the top, find the target instance on the Audit Instance page (or filter by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
4. Click Modify Shipping in the upper right corner of the Ship to TDMQ for CKafka area in the pop-up sidebar.
5. Select another CKafka instance, region, or topic (CKafka topic ID/name) in the pop-up window and click OK.
Disabling Log Shipping to CKafka
Note:
After log shipping is disabled, database audit logs of an instance will no longer be shipped. Note: After log shipping is disabled, only the shipping of new logs is stopped. The existing logs are still stored on CKafka until they expire, and storage fees will continue to be incurred during this period. If you need to delete messages, go to the CKafka console.
1. Log in to the TencentDB for MySQL console.
2. Click Database Audit in the left sidebar.
3. Select a region at the top, find the target instance on the Audit Instance page (or filter by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
4. Click Disable Shipping in the upper right corner of Ship to TDMQ for CKafka area in the pop-up sidebar.
5. Read the precautions in the pop-up window, select Disable, and click OK.
Enabling Log Shipping to COS
1. Log in to the TencentDB for MySQL console.
2. In the left sidebar, select Database Audit.
3. Select a region at the top. On the Audit Instance page, click Audit Storage Status, and select the Enabled option to filter instances with audit enabled.
4. Find the target instance in the audit instance list (or search for the instance by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
5. (Skip this step if COS has already been authorized.) In the sidebar, click Go to Authorization. Then, in the Service Authorization pop-up window, click Grant.
Note:
During the authorization process, the system will verify whether the service role authorization is successful. If the system prompts that the authorization fails, you can try authorization again later.
6. In the pop-up sidebar, click Enable Immediately below Shipping to Cloud Object Storage (COS).
7. In the Shipping to Cloud Object Storage (COS) pop-up window, complete the following configurations, and click OK.
Parameter | Description |
Target Region | Select a region for log shipping. If the region where the database instance is located is supported by COS, the region of the instance is selected by default. You can also select another available region. If the region where the database instance is located is not supported by COS, you can select another region supported by COS. |
COS Bucket | Select an existing COS bucket. The dropdown list supports quick search. If no COS bucket exists, you can select Create Bucket in the dropdown list. If you have not activated COS, the system guides you to activate it during the bucket creation process before you can proceed to complete the bucket creation operation. |
File Naming | Name the shipping file. By default, the file is named based on the shipping time. |
COS Path | Enter a COS path prefix in this field. Complete path format: prefix/year/month/day/hour. The complete path indicates the address where the audit log files are stored in the COS bucket. |
Shipping Route Example | Automatically generate a COS bucket directory based on the settings of the previous field. You can know the set COS bucket directory as displayed by this field. |
Delivery File Size | Set the shipping file size in MB. It is used together with the shipping interval. If any of the conditions are met, the file is compressed and shipped to COS according to the corresponding rule. Default value: 5. Value range: 5 to 256. For example, you set the size to 256 MB and the interval to 15 minutes. If the file size reaches 256 MB in 5 minutes, the file size condition is met, which triggers a shipping task. |
Delivery Interval Time | Specify the interval to trigger a shipping task in minutes. It is used together with the file size. If any of the conditions are met, the file is compressed and shipped to COS according to the corresponding rule. Default value: 15. Value range: 5 to 15. For example, you set the size to 256 MB and the interval to 15 minutes. If the file size is only 200 MB after 15 minutes, the shipping interval is met, which triggers a shipping task. |
Viewing Log Shipping to COS
After database audit log shipping to COS is enabled for an instance, you can view the current information on log shipping to COS (such as the COS bucket, region, and creation time for log shipping).
1. Log in to the TencentDB for MySQL console.
2. In the left sidebar, select Database Audit.
3. Select a region at the top. On the Audit Instance page, find the target instance (or search for the instance by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
4. In the pop-up sidebar, view the current log shipping information.
5. Click the COS bucket name to navigate to the file list details page of the corresponding bucket. Click Archive Storage to navigate to the COS console and view the stored shipping file.
Modifying Shipping
After database audit log shipping to COS is enabled for an instance, you can follow the steps below to modify shipping configurations as needed.
1. Log in to the TencentDB for MySQL console.
2. In the left sidebar, select Database Audit.
3. Select a region at the top. On the Audit Instance page, find the target instance (or search for the instance by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
4. In the pop-up sidebar, click Modify Delivery on the right of Shipping to Cloud Object Storage (COS).
5. In the Shipping to COS pop-up window, re-select the required configurations, and click OK.
Disabling Log Shipping to COS
Note:
After log shipping is disabled, database audit log shipping of the current instance stops. Note: After disabling, only the shipping of newly added logs stops, while logs already shipped to COS are retained until expiration. During this period, storage fees are incurred continuously. If you want to delete logs, go to the COS console for configuration.
1. Log in to the TencentDB for MySQL console.
2. In the left sidebar, select Database Audit.
3. Select a region at the top. On the Audit Instance page, find the target instance (or search for the instance by resource attributes in the search box), and choose More > Configure Log Shipping in the Operation column.
4. In the pop-up sidebar, click Disable Delivery on the right of Shipping to Cloud Object Storage (COS).
5. Read the notes in the pop-up window, select Confirm Closure, and click Confirm.
Appendix 1: Adding a Routing Policy
To ship database audit logs to TDMQ for CKafka (CKafka), you need to add a routing policy for the CKafka instance first. Otherwise, an error may occur during log shipping configuration, indicating that CKafka has no routing policy with the route type of Supporting Environment and the access mode of PLAINTEXT. Follow the steps below to add a routing policy.
1. Log in to the CKafka console.
2. Click Instance List in the left sidebar and click the ID/name of the target instance to go to the basic information page.
3. On the basic instance information page, click Add Routing Policy in the access mode module.
4. In the pop-up window, select Supporting Environment as the route type, select PLAINTEXT as the access method, and click Submit.
Appendix 2: Creating a Bucket
When enabling log shipping to COS, you need to select a COS bucket. If no COS bucket exists, you can follow the steps below to create a bucket and then select it.
1. Click Create Bucket in the dropdown list.
2. In the pop-up window, complete the following configurations, and click Create.
Parameter | Description |
Region | Select a region of the bucket. You should select a COS region corresponding to the physical region where your business is mainly located for communication with other Tencent Cloud services in the same region via the private network. The region cannot be modified after creation. |
Name | Enter a custom bucket name. Only lowercase letters, digits, and hyphens (-) are supported. The total number of characters in the domain name cannot exceed 60. The bucket name cannot be modified once set. |
Access Permission | Select the access permission. By default, a bucket is provided with three access permissions: private read/write, public read/private write, and public read/write. The permission can be modified after setting. For details, see ACL. |
Bucket Tag | Bucket tags are used as identifiers for bucket management. You can set tags for buckets to facilitate group-based bucket management. For details, see Setting Bucket Tags. |
Request Domain Name | This field displays the request domain name after the settings are completed. You can use this domain name to access the bucket. |
References
Relevant CLS documents:
Logset
Relevant CKafka documents:
Relevant COS documents:
ACL
Was this page helpful?
You can also Contact Sales or Submit a Ticket for help.
Yes
No
Feedback