LogListener service logs are used to record the operation, collection, and monitoring activities of LogListener, and you can configure visualized graphs to display such log data.
Default Configuration Item | Description |
---|---|
Log Topic | When you enable LogListener service logs, the logset cls_service_logging will be created for you automatically, and all log data generated by associated machine groups will be categorized and stored in corresponding log topics. The following three log topics are created by default: loglistener_status : the heartbeat status logs for the corresponding LogListener. loglistener_alarm : logs corresponding to LogListener's monitoring by collection metric/error type . loglistener_business : logs corresponding to LogListener's collection operations, with each log corresponds to one request. |
Region | After the LogListener log services are enabled, logsets and log topics will be created under the machine groups in the same region of LogListener. |
Log Storage Duration | The default storage duration is 7 days, and the value cannot be modified. |
Index | Full-text index and key-value index are enabled for all collected log data by default. You can modify the index configuration. For details, please see Configuring Index. |
Dashboard | Dashboard service_log_dashboard will be created in the same region of LogListener by default. |
Note:
- LogListener service logs are only about collection and monitoring activities of LogListener, and do not support writing into other kinds of data.
- Log data generated by this feature does not incur costs.
cls_service_logging
is a unified logset for LogListener service logs.
Viewing LogListener status
When the LogListener service logs are enabled, you can view LogListener running status and collection statistics. You can view the number of active LogListeners, LogListener status distribution and other statistical metrics on the service_log_dashboard
dashboard.
Log collection monitoring configuration
You can configure the collection and monitoring service logs by metric/error type. For example:
File-level monitoring
With the LogListener service logs are enabled, you can view the monitoring logs of files and directories. For example:
app1
application logs located in /var/log/app1/
. You can get the statistics of logs collected under this path.Only LogListener v2.5.4 and above support collection and monitoring service logs by machine/machine group. You’re advised to upgrade LogListener to the latest version.
Note:After this feature is disabled, the log data saved in the logset
cls_service_logging
will not be deleted automatically. You can manually delete the logset where the service logs are saved.
When the LogListener service logs are enabled, CLS will create a dashboard service_log_dashboard
by the type of recorded logs to display LogListener’s collection and monitoring statistics.
You can go to the Dashboard page of the CLS console, click the ID of the target dashboard to view LogListener collection statistics, including its status, parsing failure rate, sending success rate, and other metrics.
The parameters of the log topic loglistener_status
are detailed as follows:
Parameter | Description |
---|---|
InstanceId | LogListener unique identifier |
IP | Machine group IP |
Label | An array of machine IDs |
Version | Version number |
MemoryUsed | Memory utilization of LogListener |
MemMax | Memory utilization threshold on this machine set by the Agent |
CpuUsage | LogListener CPU utilization |
Status | LogListener running status |
TotalSendLogSize | Size of logs sent |
SendSuccessLogSize | Size of successfully sent logs |
SendFailureLogSize | Size of sending-failed logs |
SendTimeoutLogSize | Size of logs with sending timed out |
TotalParseLogCount | Total number of logs parsed |
ParseFailureLogCount | Number of parsing-failed logs |
TotalSendLogCount | Number of logs sent |
SendSuccessLogCount | Number of successful sent logs |
SendFailureLogCount | Number of sending-failed logs |
SendTimeoutLogCount | Number of logs with sending timed out |
TotalSendReqs | Total number of requests sent |
SendSuccessReqs | Number of successful sent requests |
SendFailureReqs | The number of sending-failed requests |
SendTimeoutReqs | Number of requests with sending timed out |
TotalFinishRsps | Total number of RSP files received |
TotalSuccessFromStart | Total number of successfully sent requests since LogListener was enabled |
AvgReqSize | Average request packet size |
SendAvgCost | Average sending time |
AvailConnNum | Number of available connections |
QueueSize | The size of queued requests |
The parameters of the log topic loglistener_alarm
are detailed as follows:
Monitoring Metric | Description |
---|---|
InstanceId | LogListener unique identifier |
Label | An array of machine IDs |
IP | Machine group IP |
Version | LogListener version |
AlarmType.count | Statistics of alarm types |
AlarmType.example | Sample alarm type |
AlarmType:
alarm type | type ID | Description |
---|---|---|
UnknownError | 0 | Initializing the alarm type. |
UnknownError | 1 | Failed to parse. |
CredInvalid | 2 | Failed to verify. |
SendFailure | 3 | Failed to send. |
RunException | 4 | Abnormal LogListener running. |
MemLimited | 5 | Reached the memory utilization threshold. |
FileProcException | 6 | Exceptions occurred in file processing. |
FilePosGetError | 7 | Failed to get the file publishing info. |
HostIpException | 8 | Exceptions occurred in the server IP thread. |
StatException | 9 | Failed to get the process info. |
UpdateException | 10 | Exceptions occurred in the CLS modification feature. |
DoSendError | 11 | Failed to confirm sending. |
FileAddError | 12 | Failed to create the file. |
FileMetaError | 13 | Failed to create the metadata file. |
FileOpenError | 14 | Failed to open the file. |
FileOpenError | 15 | Failed to read the file. |
FileStatError | 16 | Failed to get the file status. |
getTimeError | 17 | Failed to get the time from the log content. |
HandleEventError | 18 | Exceptions occurred in processing the file. |
handleFileCreateError | 19 | Exceptions occurred in handleFileCreateEvent() . |
LineParseError | 20 | Failed to parse the log directory. |
Lz4CompressError | 21 | Failed to compress. |
readEventException | 22 | Failed to read. |
ReadFileBugOn | 23 | A bug exists. |
ReadFileException | 24 | Exceptions occurred in the read file. |
ReadFileInodeChange | 25 | File node changed. |
ReadFileTruncate | 26 | The read file is truncated. |
WildCardPathException | 27 | Exceptions occurred in addWildcardPathInotify() . |
The parameters of the log topic loglistener_business
are detailed as follows:
Parameter | Description |
---|---|
InstanceId | LogListener unique identifier |
Label | An array of machine IDs |
IP | Machine group IP |
Version | LogListener version |
TopicId | The target topic of the collected file |
FileName | File path name |
FileName | Actual file path |
FileInode | File node |
FileSize | File size |
LastReadTime | The most recent read time of the file |
ParseFailLines | Number of parsing-failed logs within a time window |
ParseFailSize | Size of parsing-failed logs within a time window |
ParseSuccessLines | Number of logs successful parsed within a time window |
ParseSuccessSize | Size of logs successful parsed within a time window |
ReadOffset | Offset of file reading in bytes |
TruncateSize | Size of truncated log files within a time window |
ReadAvgDelay | Average time delay for reads within a time window |
TimeFormatFailuresLines | Number of timestamp matching errors within a time window |
SendSuccessSize | Size of logs successful sent within a time window |
SendSuccessCount | Number of logs successful sent within a time window |
SendFailureSize | Size of sending-failed logs within a time window |
SendFailureCount | Number of sending-failed logs within a time window |
SendTimeoutSize | Size of logs with sending timed out within a time window |
SendTimeoutCount | Number of logs with sending timed out within a time window |
DroppedLogSize | Size of dropped logs within a time window |
DroppedLogCount | Number of dropped logs in a time window |
ProcessBlock | Whether the current file has triggered collection blocking in a statistical period (collection blocking will be triggered if the sliding window of a file has not moved for 10 minutes) |
Was this page helpful?