The Event Center of Cloud Monitor (CM) currently provides the following monitoring information for product events:
| Event Name |
Event Parameter |
Event Type |
Dimension | Recoverable |
Description | Solution and Suggestion |
|---|---|---|---|---|---|---|
| Kernel failure | GuestCoreError | Exception |
CVM instance | No | An OS kernel bug or driver issue causes a fatal error in the OS kernel | 1. Check whether any kernel driver modules are loaded into the system other than those provided by the kernel. Try not to load these modules and observe the operating status of the system. 2. View the bug reports of the kernel and OS, and try to upgrade the kernel. 3. By default, kdump is enabled for CVM. When a panic occurs, system memory dump information will be generated in the /var/crash directory. You can analyze it with the crash tool |
| OOM | GuestOom | Exception |
CVM instance | No | System memory usage is overloaded | 1. Check whether the memory capacity configured in the current system meets business requirements. If additional capacity is required, we recommend you upgrade the CVM memory configuration. 2. View processes that are killed during OOM based on system logs such as dmesg and /var/log/messages to check whether the process memory is used as expected. Use tools such as valgrind to analyze whether memory leakage occurs |
| ping failure | PingUnreachable | Exception |
CVM instance | Yes | The network of the CVM instance is unpingable | 1. Check whether the running status of the CVM instance is normal. If any exception occurs (for example, the system crashes), force restart the CVM instance in the console to recover it. 2. If the CVM instance is running normally, check the CVM network configuration, including the internal network service configuration, firewall configuration, security group configuration, etc. |
| Read-only disk | DiskReadonly | Exception |
CVM instance | Yes | Data cannot be written into the disk | 1. Check whether the disk is full. 2. On Linux, run the `df -i` command to check whether inode is used up. 3. Check whether the file system is corrupted |
| Server restart | GuestReboot | Status change |
CVM instance | Yes | The CVM instance restarts | This event is triggered when the CVM restarts. Check whether the status change is as expected based on actual scenarios |
| Packet loss occurs when the public network outbound bandwidth exceeds the limit | PacketDroppedByQosWanOutBandwidth | Exception |
CVM instance | Yes | The public network outbound bandwidth of the CVM instance exceeds the upper limit, causing packet loss. Packet loss caused by bandwidth glitches is not reflected on the bandwidth view, because the minimum granularity for bandwidth statistics is 10 (total traffic in 10 seconds/10 seconds). If the constant bandwidth is not exceeded by much, it can be ignored | Increase the upper limit for public network bandwidth. If the maximum limit can be purchased has been reached, you can reduce the bandwidth consumption of the server through load balancing and other means |
| Packet loss occurs when the number of connections exceeds the limit | PacketDroppedByQosConnectionSession | Exception |
CVM instance | Yes | There are too many connections to the CVM instance, causing packet loss | Contact after sales service |
| Event Name |
Event Parameter |
Event Type |
Dimension | Recoverable |
Description | Solution and Suggestion |
|---|---|---|---|---|---|---|
| Blocked public IP | VipBlockInfo | Exception |
CLB instance | Yes | The CLB public IP under attack is blocked after an exception is detected by the security protection system | Submit a ticket for specific causes and solutions |
| Server port status has an exception | RsPortStatusChange | Exception |
Real server port | Yes | An exception is found at the real server port of the public network CLB instance during health check | View the service status of the real server port |
| Event Name |
Event Parameter |
Event Type |
Dimension | Recoverable |
Description | Solution and Suggestion |
|---|---|---|---|---|---|---|
| Packet loss occurs when the public network outbound bandwidth exceeds the limit | PacketDroppedByQosWanOutBandwidth | Exception |
VPN Gateway instance | Yes | The public network outbound bandwidth of the VPN Gateway instance exceeds the upper limit, causing packet loss. Packet loss caused by bandwidth glitches is not reflected on the bandwidth view, because the minimum granularity for bandwidth statistics is 10 (total traffic in 10 seconds/10 seconds). If the constant bandwidth is not exceeded by much, it can be ignored | Increase the upper limit for public network bandwidth |
| Packet loss occurs when the number of connections exceeds the limit | PacketDroppedByQosConnectionSession | Exception |
VPN Gateway instance | Yes | There are too many connections to the VPN Gateway instance, causing packet loss | Contact after sales service |
| Event Name |
Event Parameter |
Event Type |
Dimension | Recoverable |
Description | Solution and Suggestion |
|---|---|---|---|---|---|---|
| Node exception | NodeNotReady | Exception |
Cluster | Yes | A node exception may be caused by a variety of reasons, such as network disconnection, node kubelet exception, and container OOM. If the node has an exception for a long time, Kubernetes will drain the containers on the node | 1. Check whether the node is running on the CVM monitoring page. 2. Log in to the node server to check whether kubelet is running normally. 3. Log in to the node server to check whether docker is running normally |
| Node disk capacity will run out soon | NodeHasDiskPressure | Exception |
Cluster | Yes | The disk (cbs or root) capacity used for container and image storage on the node will run out soon. NodeOutOfDisk will be triggered after the capacity runs out, and new containers cannot be scheduled to this node | Clean up the disk or container images no longer in use |
| Node disk capacity has run out | NodeOutOfDisk | Exception |
Cluster | Yes | The disk (cbs or root) capacity used for container and image storage on the node has run out, and new containers cannot be scheduled to this node | Clean up the disk or container images no longer in use |
| Insufficient node memory | NodeHasInsufficientMemory | Exception |
Cluster | Yes | The node memory utilization is high | Expand the node or schedule containers to other nodes |
| Node OOM | SystemOOM | Exception |
Cluster | No | OOM occurs on the node due to high memory utilization | Check the cause of OOM on the node by viewing monitoring data, syslog, demsg, etc. |
| Unreachable node network | NodeNetworkUnavailable | Exception |
Cluster | No | The network on the node is not configured properly. Normally, this problem will not occur to clusters created through the console or Tencent Cloud API | Submit a ticket or contact after sales service |
| Insufficient inodes on the node | NodeInodePressure | Exception |
Cluster | No | Insufficient inodes on the node will prevent the node from creating containers | Check the remaining inodes on the node. Try to clean up container images no longer in use to free up inode space |
| Event Name |
Event Parameter |
Event Type |
Dimension | Recoverable |
Description | Solution and Suggestion |
|---|---|---|---|---|---|---|
| OOM | OutOfMemory | Exception |
TencentDB for MySQL instance | Yes | Database memory usage is overloaded | Check whether the memory capacity configured in the database meets business requirements. If additional capacity is required, we recommend you upgrade the MySQL memory configuration |
| Master/slave switchover | PrimarySwitch | Exception |
TencentDB for MySQL instance | No | Master/slave switchover occurs | This event may be triggered when a physical machine fails. Check whether the instance status is normal |
| Removal of read-only instance | RORemoval | Exception |
TencentDB for MySQL instance | Yes | A read-only instance fails or exceeds the latency threshold | If the read-only group contains only one read-only instance, promptly switch the read traffic after deleting this read-only instance to avoid single points of failure. We recommend you purchase at least two read-only instances for the group |
| Instance migration caused by server failure | ServerfailureInstanceMigration | Exception |
TencentDB for MySQL instance | No | Instance migration is caused by a server failure | The migration time is subject to the maintenance window. Please change the time promptly if needed. The new migration time will be subject to the new maintenance window |
| Event Name |
Event Parameter |
Event Type |
Dimension | Recoverable |
Description | Solution and Suggestion |
|---|---|---|---|---|---|---|
| Insufficient backup oplog | oplogInsufficient | Exception |
TencentDB for MongoDB instance | No | When TencentDB for MongoDB is backing up data, it cannot read the full oplog from the last backup to the current backup. This affects database rollback to any time point in the last 7 days | We recommend you adjust the size or backup frequency of MongoDB oplog on the TencentDB for MongoDB Console. If you do not need this event notification, you can disable it on the backup page in the console |
| The number of connections exceeds the limit | connectionOverlimit | Exception |
TencentDB for MongoDB instance | Yes | The number of connections to the instance exceeds the limit | Check whether the maximum number of connections configured for the TencentDB for MongoDB instance meets business requirements. If additional connections are required, we recommend you upgrade the instance configuration |
| Master/slave switchover | primarywitch | Exception |
TencentDB for MongoDB instance | Yes | Switchover occurs between primary and secondary databases | This event may be triggered when a physical machine fails. Check whether the instance status is normal |
| The disk capacity has run out | instanceOutOfDisk | Exception |
TencentDB for MongoDB instance | Yes | The disk capacity is full and the instance becomes read-only | Clean up the disk |
| Instance rollback | instanceRollback | Exception |
TencentDB for MongoDB instance | Yes | The instance data is rolled back | This event may be triggered if the master node fails and a master/slave switchover occurs when some data on the master node has not been synced to the slave node in time. Check whether the instance status is normal |
| Event Name |
Event Parameter |
Event Type |
Dimension | Recoverable |
Description | Solution and Suggestion |
|---|---|---|---|---|---|---|
| Connection downtime | DirectConnecDown | Exception |
Connection | Yes | The physical linkage transmission of the connection is interrupted or has an exception | 1. Check whether the physical line has an exception or is interrupted (for example, the fiber cable is cut off, the line is unplugged, etc.) 2. Check whether the receiving port and optical/electrical modules are functioning normally 3. Check whether the network device port is turned off |
| Dedicated tunnel downtime | DirectConnectTunnelDown | Exception |
Dedicated tunnel | Yes | The physical linkage transmission of the connection is interrupted or exceptional | 1. Check whether the physical line has an exception or is interrupted (for example, the fiber cable is cut off, the line is unplugged, etc.) 2. Check whether the receiving port and optical/electrical modules are functioning normally 3. Check whether the network device port is turned off |
| Dedicated tunnel BGP session downtime | DirectConnectTunnelBGPSessionDown | Exception |
Dedicated tunnel | Yes | The dedicated tunnel BGP session is interrupted | 1. Check whether the BGP process of the network device is normal 2. Check whether the dedicated tunnel is normal 3. Check whether the physical line is normal |
| Alarm for excessive BGP tunnel routes | DirectConnectTunnelRouteTableOverload | Exception |
Dedicated tunnel | No | The number of BGP session routes in a dedicated tunnel exceeds 80% of the threshold | 1. Check whether the routes published by the dedicated tunnel's BGP session reaches 80% of the upper limit (the default value is 100. For more information, please see here) |
Was this page helpful?