A log with full text in a single line means a line is a full log. When CLS collects logs, it uses the line break \n
to mark the end of a log. For easier structural management, a default key value __CONTENT__
is given to each log, but the log data itself will no longer be structured, nor will the log field be extracted. The time attribute of a log is determined by the collection time.
Assume that the raw data of a log is as follows:
Tue Jan 22 12:08:15 CST 2019 Installed: libjpeg-turbo-static-1.2.90-6.el7.x86_64
The log is eventually structured by CLS as follows:
__CONTENT__:Tue Jan 22 12:08:15 CST 2019 Installed: libjpeg-turbo-static-1.2.90-6.el7.x86_64
test_full
as Log Topic Name and click OK.On the Collection Configuration page, set Collection Path according to the log collection path format as shown below:
Log collection path format: [directory prefix expression]/**/[filename expression]
.
After the log collection path is entered, LogListener will match all common prefix paths that meet the [directory prefix expression] rule and listen for all log files in the directories (including subdirectories) that meet the [filename expression] rule. The parameters are as detailed below:
Parameter | Description |
---|---|
Directory prefix | Directory structure of the log file prefix. Only wildcards \* and ? are supported.
|
/**/ | Current directory and all its subdirectories. |
Filename | Log filename. Only wildcards \* and ? are supported.
|
Common configuration modes are as follows:
Below are examples:
No. | Directory Prefix Expression | Filename Expression | Description |
---|---|---|---|
1. | /var/log/nginx | access.log | In this example, the log path is configured as /var/log/nginx/**/access.log . LogListener will listen for log files named access.log in all subdirectories in the /var/log/nginx prefix path. |
2. | /var/log/nginx | *.log | In this example, the log path is configured as /var/log/nginx/**/*.log . LogListener will listen for log files suffixed with .log in all subdirectories in the /var/log/nginx prefix path. |
3. | /var/log/nginx | error* | In this example, the log path is configured as /var/log/nginx/**/error* . LogListener will listen for log files prefixed with error in all subdirectories in the /var/log/nginx prefix path. |
Note:
- Only LogListener 2.3.9 and above support adding multiple collection paths.
- The system does not support uploading logs with contents in multiple text formats, which may cause write failures, such as
key:"{"substream":XXX}"
.- You are advised to configure the collection path as
log/*.log
and rename the old file after log rotation aslog/*.log.xxxx
.- By default, a log file can only be collected by one log topic. If you want to have multiple collection configurations for the same file, please add a soft link to the source file and add it to another collection configuration.
In the Collection Configuration page, select Full text in a single line as the Extraction Mode.
Filters are designed to help you extract valuable log data by adding log collection filter rules based on your business needs. If the filter rule is a Perl regular expression, the created filter rule will be used for matching; in other words, only logs that match the regular expression will be collected and reported.
By default, this "full text in a single line" mode uses __CONTENT__
as the key name of a log. Assume that a sample log is Tue Jan 22 12:08:15 CST 2019 Installed: libjpeg-turbo-static-1.2.90-6.el7.x86_64
, and you want to collect all logs on Jan 22, then enter __CONTENT__
in Key and Tue Jan 22.*
in Filter Rule.
Note:The relationship logic between multiple filter rules is "AND". If multiple filter rules are configured for the same key name, previous rules will be overwritten.
Click Next to go to the Index Configuration page.
On the Index Configuration page, set the following information:
@&()='",;:<>[]{}/ \n\t\r
and can be modified as needed.Note:Index configuration must be enabled before you can perform searches.
Click OK to finish collection configuration.
Was this page helpful?