Item | Capability and Limit |
File encoding | UTF8 and GBK encoding formats are supported. Note: The GBK encoding format requires LogListener 2.6.2 or later. |
Soft link | Supported. |
Size of a single log | The size of a single log line is limited to 1 MB. If a log exceeds 1 MB, it is truncated, retaining only the first 1 MB. After multi-line logs are split based on the line-start regular expression (when the first line is determined, data before \\n is read), a single log is limited to a maximum of 2 MB. If a log exceeds 2 MB, it is truncated, retaining only the first 2 MB. |
Regular expression | Perl-compatible regular expressions are supported. |
First log collection behavior | LogListener supports full and incremental collection policies: Full collection: Upon initial installation and startup, LogListener collects all logs that meet the specified conditions, including any unwritten files. Incremental collection: Upon initial installation and startup, LogListener collects existing files starting from the most recent position. Note: Both incremental collection and full collection require LogListener 2.6.2 or later. |
Log file rotation | Supported. (It is recommended that the file name after rotation should not be overwritten by the collection wildcard path.) Note: If no new log entries are written to a rotated log file within 60 seconds after rotation, it will not be collected again. Otherwise, the collection of the rotated log file will continue. |
Collection behavior when log parsing fails | We recommend you enable the feature of uploading parsing-failure logs. If the feature is enabled, parsing-failure logs will be uploaded to the preset index in the format of full text in a single line. Otherwise, the logs will be discarded. |
File opening | LogListener opens the file when it starts collecting and reading, and closes it after the reading is complete. |
Filter rule. | Only one filter rule can be configured for both single-line and multi-line full-text collection operations. |
| Up to five filter rules can be configured for delimiter, regular expression, and JSON parsing modes. |
Key field length | The maximum length for a Key field is 128 bytes for both delimiter and regular expression parsing modes. |
Number of key-value pairs | In all parsing modes, the maximum number of key-value pairs that can be extracted is 1,000. Any excess will be discarded. |
LogListener maximum connections | It is limited to 1,024. |
Memory usage | The memory usage of LogListener can be up to 50 MB under normal conditions. |
CPU usage | Under a log volume of 5 MB/s, the total CPU usage of LogListener's business processes does not exceed 20% of a single core. |
Collection configuration activation delay | There will be a 1-minute delay before the collection configuration takes effect after it is updated. |
Log file | It is recommended that the size of the log file targeted by LogListener for collection be less than 500 MB. |
Item | Capability and Limit |
Checkpoint storage location | For LogListener < 3.4.0: The default storage path is the data directory under the LogListener installation directory. You can modify this path using the checkpoint_cache_file parameter in the LogListener configuration file. For LogListener ≥ 3.4.0: The storage path is fixed to the data/checkpoint directory under the LogListener installation directory and cannot be modified through configuration parameters. |
Checkpoint storage policy | LogListener stores two copies of the checkpoint metadata: One copy records the information only of checkpoints where upload is completed, and the information is persisted to a disk in real time. The other copy records the information of checkpoints where upload is completed and checkpoints where upload is not completed, and the information is periodically persisted to a disk. Persistence takes precedence when the program exits. |
Item | Capability and Limit |
Default CPU core limit | LogListener is designed to operate on a single core only. |
Default CPU resource limits | LogListener does not impose a CPU resource limit by default. To set a limit, refer to LogListener Configuration Guide and modify the cpu_usage_thres parameter in the LogListener configuration file to specify the maximum CPU utilization. Under the current LogListener architecture, if no CPU resource limit is set, its maximum achievable single-core CPU utilization is approximately 110% (business threads up to 100%, management threads around 10%). |
Default thread count limit | LogListener 2.x.x edition supports single-threaded operation only. LogListener 3.x.x edition supports multi-threading capability. |
Default memory resource limits | LogListener has a default memory threshold of 2 GB. To set a limit, refer to LogListener Configuration Guide and modify the max_mem parameter in the LogListener configuration file. Modify the max_mem parameter to set the maximum memory usage. The recommended value is not less than 300 MB. |
Default bandwidth resource limits | LogListener does not impose a bandwidth resource limit by default. To set a limit, refer to LogListener Configuration Guide and modify the max_send_rate parameter in the LogListener configuration file to limit the network bandwidth used by the program. |
Resource overrun handling policy | If LogListener consumes CPU and memory resources beyond the maximum limit for more than five minutes, the collection program will automatically restart. |
Log compression | The collected logs are uploaded in a compressed format by default. If you do not require compression, modify request_compression in the LogListener configuration file by reference to LogListener Configuration Guide to disable compressed uploads. |
Number of monitored directories | The default maximum number of monitored directories is 5,000. Exceeding this limit may lead to collection failures. |
Number of monitored files | The recommended maximum number of monitored files is 10,000. If this limit is exceeded, log collection may fail. |
Maximum number of monitored events | In polling mode, the recommended maximum number of monitored events is 10,000. Exceeding this limit may lead to collection failures. |
Item | Capability and Limit |
Network error handling | Except exceptions requiring special handling (such as log topic deletion), all other errors (such as network exceptions, timeout, frequency control, and overdue payment) will be retried. |
Maximum retry timeout duration | If data fails to be sent after retrying for more than 1 hour, the system discards the data. The default behavior is to retry at an interval, and the retry interval becomes longer and longer until the maximum retry timeout duration is exceeded. |
Maximum number of retries | The maximum number of retries can be set in the If the maximum number of retries is set, the system retries until the maximum number of retries is exceeded, and then discards the data. |
Line break missing | During collection, if a single log entry lacks a line break at its end, the collector waits for five minutes. If no line break is detected within five minutes, it automatically appends a line break and continues collecting. |
Item | Capability and Limit |
Log upload policy | LogListener automatically aggregates and uploads logs from the same file when any of the following conditions are met: 10,000 log entries, a total logset size of 1 MB, or a log collection time exceeding three seconds. Once any of these conditions is fulfilled, the upload process is initiated. |
File collection handling policy | A single target file (any file matching a collection path) can only be uploaded to one log topic. Multiple collection paths cannot point to the same file. To upload a single file to multiple log topics, create multiple soft links to that file, and configure each log topic to collect from its own soft link. |
Log collection delay | In the case of real-time collection, data collection, transmission, and storage to disks will be completed within 1 minute, and the logs are retrievable in the console. If the log volume is large, or LogListener can use only a limited amount of resources, there will be some collection delay. |
Item | Capability and Limit |
Maximum number of collection configurations | The maximum number of collection configurations that can be associated with one log topic is 100. |
Item | Capability and Limit |
Machine group logic | Currently, machine groups are divided into two categories. The usage methods for these two categories are independent of each other and are incompatible. If they are used together, the collection machine will fail to pull the correct collection configuration, resulting in a failure to collect data. IP machine group: a machine IP must be manually added to the machine group in the console, and the Label machine group: the machine group label is set in the console, and the |
Relationships between machine groups and log topics | A single log topic can be bound to multiple machine groups. A single machine group can be bound to multiple log topics. |
Relationships between machine groups and machines | A single machine can be added to multiple machine groups. For an IP machine group, there is no limit to the number of machine groups that a machine can join. For a label machine group, the maximum number of machine groups that a machine can join is 20. |
Limits on the labels of label machine groups | The label for a Tag machine group is currently limited to 32 characters. Up to 20 labels can be configured for a single label machine group. |
Item | Capability and Limit |
Collection blocklist | This feature is used to specify the content to ignore in the collection path. Currently, collection blocklists support two modes: FILE mode: Specify the files to ignore in the collection path. Wildcard filtering is supported. PATH mode: Specify the subdirectories to ignore in the collection path. Wildcard filtering is supported. Note: The FILE and PATH modes can be used together. The collection blocklist excludes paths under the collection path. Therefore, in both file name mode and directory mode, the specified path should be a subset of the collection path. |
フィードバック