A log with full text in multi lines spans multiple lines (such as a Java program log). In this mode, the line break \n
cannot be used to mark the end of a log. To help CLS system distinguish between the logs, a first-line regular expression is used for match. When a log in a line matches the preset regular expression, it is considered as the beginning of a log, and the next matching line will be the end mark of the log.
In the "full text in multi lines" mode, a default key __CONTENT__
is also set, but the log data itself is not structured, and no log fields are extracted. The time attribute of a log is determined by the collection time.
Assume the raw data of a multi-line log is:
10.20.20.10 - - [Tue Jan 22 14:24:03 CST 2019 +0800] GET /online/sample HTTP/1.1 127.0.0.1 200 628 35 http://127.0.0.1/group/1
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0 0.310 0.310
CLS outputs it into:
__CONTENT__:10.20.20.10 - - [Tue Jan 22 14:24:03 CST 2019 +0800] GET /online/sample HTTP/1.1 127.0.0.1 200 628 35 http://127.0.0.1/group/1 \nMozilla/5.0 (Windows NT 10.0; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0 0.310 0.310
test-mtext
as the Log Topic Name and click OK.On the Collection Configuration page, enter a Collection Path according to the log collection path format as shown below:
Log collection path format: [directory prefix expression]/**/[filename expression]
.
After the log collection path is entered, LogListener will match all common prefix paths that meet the [directory prefix expression] rule and listen for all log files in the directories (including subdirectories) that meet the [filename expression] rule. The parameters are as detailed below:
Field | Description |
---|---|
Directory prefix | Directory structure of the log file prefix. Only wildcards \* and ? are supported.
|
/**/ | Current directory and all its subdirectories |
Filename | Log filename. Only wildcards \* and ? are supported.
|
Common configuration modes are as follows:
Below is an example:
No. | Directory Prefix Expression | Filename Expression | Description |
---|---|---|---|
1. | /var/log/nginx | access.log | In this example, the log path is configured as /var/log/nginx/**/access.log . LogListener will listen for log files named access.log in all subdirectories in the /var/log/nginx prefix path. |
2. | /var/log/nginx | *.log | In this example, the log path is configured as /var/log/nginx/**/*.log . LogListener will listen for log files suffixed with .log in all subdirectories in the /var/log/nginx prefix path. |
3. | /var/log/nginx | error* | In this example, the log path is configured as /var/log/nginx/**/error* . LogListener will listen for log files prefixed with error in all subdirectories in the /var/log/nginx prefix path. |
Note:
- Only LogListener 2.3.9 or above allows adding multiple collection paths.
- By default, a log file can only be collected by one log topic. If you want to have multiple collection configurations for the same file, please add a soft link to the source file and add it to another collection configuration.
In the Collection Configuration page, select Full text in multi lines as the Extraction Mode.
Define a regular expression according to the following rules.
You can choose Auto-Generate or Enter Manually to define a first-line regular expression, and the system will verify the regular expression based on the sample content.
Auto-Generate: enter the sample log in the text box, click Auto-Generate, and the system will automatically generate the first-line regular expression in the grayed-out text box as shown below:
Enter Manually: enter the sample log and first-line regular expression in the text box, click Verify, and the system will determine whether the expression has passed verification as shown below:
Filters are designed to help you extract valuable log data by adding log collection filter rules based on your business needs. If the filter rule is a Perl regular expression, the created filter rule will be a hit rule; in other words, only logs that match the regular expression will be collected and reported.
In the "full text in multi lines" mode, __CONTENT__
is used as the key name of a log by default. For example, below is a sample log with full text in multi lines:
10.20.20.10 - - [Tue Jan 22 14:24:03 CST 2019 +0800] GET /online/sample HTTP/1.1 127.0.0.1 200 628 35 http://127.0.0.1/group/1
Mozilla/5.0 (Windows NT 10.0; WOW64; rv:64.0) Gecko/20100101 Firefox/64.0 0.310 0.310
If you want to collect all logs of the server 10.20.20.10
, enter __CONTENT__
in Key and 10.20.20.10.*
in Filter Rule.
Note:
The relationship logic between multiple filter rules is "AND". If multiple filter rules are configured for the same key name, previous rules will be overwritten.
@&()='",;:<>[]{}/ \n\t\r
by default and can be modified as needed.Note:
Index configuration must be enabled before you can perform searches.
Was this page helpful?