Through configuration in the CLS console, you can ship log data to COS in the JSON format. This document describes how to create a JSON log shipping task.
The parameters are described as follows:
Configuration Item | Description | Format | Required |
---|---|---|---|
Shipping Task Name | Name of a shipping task | Can contain letters, numbers, underscores (_), and hyphens (-) | Yes |
COS Bucket | Target bucket for shipping. The target bucket must be in the same region as the current log topic. | Select from drop-down list | Yes |
Directory Prefix | Prefix of the directory under the specified COS bucket to which log files are shipped. You can customize a directory prefix. By default, log files are stored directly under the specified bucket as {COS bucket}{directory prefix}{partition format}_{random}_{index}.{type} , where {random}_{index} is a random number. |
Cannot start with / |
No |
Partition Format | A string converted from the creation time of the shipping task using the strftime function. / indicates directory levels. |
Must be in strftime format | Yes |
File Size | Maximum size of an uncompressed file to be shipped during the specified shipping interval. A log file larger than this size will be split into multiple files. The value can be from 100 MB to 256 MB. | Value range: 100-256 (MB) | Yes |
Shipping Interval | Time interval of shipping, which can be 300-900 seconds. If you set it to 5 minutes, a log file will be generated from your log data every 5 minutes, and multiple log files will be shipped to your bucket at a time (with an interval of less than 30 minutes). | Value range: 300-900 (s) | Yes |
logset/
, and the shipping time `2018/7/31 17:14`, the path to which log files are saved would vary with the partition format entered, as shown below:
Bucket | Directory Prefix | Partition Format | COS File Path |
---|---|---|---|
bucket_test | logset/ | %Y/%m/%d | bucket_test:logset/2018/7/31_{random}_{index} |
bucket_test | logset/ | %Y%m%d/%H | bucket_test:logset/20180731/14_{random}_{index} |
bucket_test | logset/ | %Y%m%d/log | bucket_test:logset/20180731/log_{random}_{index} |
The parameters are described as follows:
Configuration Item | Description | Format | Required |
---|---|---|---|
Compressed Shipping | Whether to compress log files before shipping. The maximum size of an uncompressed file to be shipped is 10 GB. Log files can be compressed in GZip, Lzop, or Snappy format. | Switch | Yes |
Was this page helpful?