LogListener Updates

Last updated: 2021-10-25 15:04:18

This document describes CLS LogListener updates.

Note:

  • Full/Incremental collection policies are available starting from LogListener v2.6.2.
  • CVM batch deployment is available starting from LogListener v2.6.0.
  • Multi-line - full regular expression collection mode is available starting from LogListener v2.4.5.
  • LogListener auto upgrade is available starting from LogListener v2.5.0.
  • Uploading parsing-failed logs is available starting from LogListener v2.5.2.
  • You are advised to install or upgrade to the latest version for a better user experience.
VersionChange TypeDescription
v2.6.2New featureAdded support for incremental collection.
Experience optimization
  • Optimized the issue where collection is ignored in the period from file scanning to processing.
  • Optimized abnormal overriding during automatic upgrade.
v2.6.1Experience optimization
  • Optimized the issue where backtracking collection may occur during log rotation in some scenarios.
  • Adjusted the timeout duration for log upload on the collection end to avoid data duplication caused by timeout.
v2.6.0New feature
  • Added support for CVM batch deployment.
  • Added support for ciphertext storage of secret IDs/KEYs.
Experience optimization
  • Optimized the LogListener installation and stop logic.
  • Optimized the retry policy upon upload failures.
  • Added a tool for detecting and rectifying dead locks caused by Glibc libraries of earlier versions.
  • Optimized collection performance.
v2.5.9Experience optimizationOptimized the resource limit policy.
v2.5.8Experience optimization
  • Fixed the issue that removing a directory soft link affects the collection of other directory soft links that point to the same target.
  • Fixed the issue that files in a directory cannot be collected if a soft link of the directory is removed and the same soft link is created again.
v2.5.7Experience optimization
  • Fixed the (new) issue that logs will be collected again when the log file size is greater than 2 GB.
  • Fixed the issue where renaming too many files will cause the program to malfunction.
  • Fixed the issue where specified fields cannot be updated under log collection monitoring.
v2.5.6Experience optimizationOptimized the issue that under specific use cases, the collection program cannot be triggered.
v2.5.5Experience optimization
  • Optimized metadata checkpoints for collection to guarantee no data will lose due to restart.
  • Supports resource limit configuration and overrun handling for memory, CPU, and bandwidth.
v2.5.4New featureAdded support for log collection monitoring.
Experience optimizationEnhanced memory overrun handling: LogListener will be automatically loaded when memory overrun lasts for a period of time.
v2.5.3Experience optimizationOptimized LogListener exceptions caused by memory issues.
v2.5.2New featureAdded support for uploading parsing-failed logs.
Experience optimizationOptimized the blocklist feature. Now, the blocklist FILE mode supports wildcard filtering.
v2.5.1Experience optimizationEnhanced the handling when breakpoint metadata could not be found in the collection file.
v2.5.0New feature
  • Added support for automatic LogListener upgrade.
  • Added support for automatic LogListener start in Ubuntu operating system.
v2.4.6Experience optimization
  • Cleared residual configuration data in the cache after the collection configuration was changed.
  • Optimized the issue where file collection with a soft link pointing to the `realpath` file was affected when an `IN_DELETE` event that deleted the soft link was being processed.
  • Optimized the feature of collecting the same source file via the file's soft link and the directory's soft link at the same time.
v2.4.5New featureAdded support for `multiline_fullregex_log` log collection.
v2.4.4Experience optimizationOptimized the issue of inaccurate log time caused by the msec feature.
v2.4.3New featureAdded support for automatically checking the log format (logFormat).
v2.4.2Experience optimizationOptimized the issue of cache eviction during configuration pulling in Tencent Cloud container scenarios.
v2.4.1New featureAdded support for collecting logs in milliseconds.
Experience optimizationOptimized exceptions due to no line break data in user logs.
v2.4.0New featureAdded support for instance-level process monitoring by LogListener.
v2.3.9New featureAdded support for blocklisting collection paths.
Experience optimizationOptimized the memory leak issue due to outdated Boost library.
v2.3.8New featureAdded support for multi-path log collection.
v2.3.6Experience optimization
  • Fixed the issue where collection stopped due to invalid key value.
  • Fixed the memory leak issue due to request failures with the error code 502 returned.
v2.3.5New featureAdded support for log context search.
Experience optimization
  • Fixed the issue where log collection stopped when logs were uploaded but authentication failed in the static configuration mode.
  • Fixed the issue where dynamic configurations were no longer read after the memory exceeded the threshold in the dynamic configuration mode.
  • Fixed the issue where sometimes log collection repeated when the log production speed was too high during log rotation.
  • Fixed the memory leak issue caused by multiple failures to upload logs.
v2.3.1Experience optimization
  • Optimized memory limit.
  • When the memory limit was reached, requests lasting over 3s were considered as timed out.
v2.2.6New featureAdded support for configuring private domain names and public domain names separately.
Experience optimizationFixed LogListener exceptions caused by `getip`.
v2.2.5New featureAdded support for Tencent Cloud COC environment deployment.
Experience optimizationFixed the core issue caused by `getip`.
v2.2.4Experience optimization
  • Changed the commands for installation and initialization to the subcommands `install` and `init` of `tools/loglistener.sh` respectively.
  • Changed the command for restart to `/etc/init.d/loglistenerd start|stop|restart`.
v2.2.3Experience optimizationRenaming or creating logs during log rotation will not cause log loss.
v2.2.2Experience optimizationA log greater than 512 KB will be automatically truncated.
Earlier versions-
  • v2.2.2 added support for collection by full regular expression.
  • v2.1.4 added support for full text in multi lines.
  • v2.1.1 added support for log structuring.