Before the CoreDNS component is upgraded, the platform automatically performs a pre-check to detect configuration issues that may affect the upgrade. The pre-check results are categorized into two levels:
Fatal: The upgrade is blocked and must be resolved before proceeding.
Warning: This does not block the upgrade. It is recommended to address the issue.
The following sections explain the meaning and handling methods of each pre-check item, categorized by the object that needs to be modified.
Corefile Configuration
The following pre-check items require modifications to the CoreDNS Corefile configuration. Procedure:
kubectl edit configmap coredns -n kube-system
1. Corefile Not Found (Fatal)
Pre-check Item: Coredns_Corefile_Not_Exist
Description: The upgrade cannot proceed because the CoreDNS ConfigMap lacks the Corefile configuration item.
Solution: Verify that a ConfigMap named coredns exists in the kube-system namespace and contains the Corefile field. If it is missing, manually add the standard Corefile configuration.
2. Corefile Syntax Error (Fatal)
Pre-check Item: CoreDNS_Corefile_Format_Error
Description: The Corefile contains syntax errors and cannot be parsed correctly.
Solution: Check the Corefile syntax. Common issues include mismatched braces, incorrect indentation, misspelled directives, and so on.
3. Missing health Plugin (Fatal)
Pre-check Item: Coredns_Plugin_Health_Not_Exist
Description: The health plugin is not configured in the Corefile. CoreDNS requires the health plugin to provide a health check endpoint. The absence of this plugin will cause Pod liveness checks to fail after the upgrade, resulting in repeated restarts.
Solution: Add the health plugin to the server block in the Corefile:
.:53 {
health
ready
kubernetes cluster.local in-addr.arpa ip6.arpa {
...
}
forward . /etc/resolv.conf
...
}
4. Coexistence of proxy and forward Plugins (Fatal)
Pre-check Item: Coredns_Plugin_Proxy_Forward_Coexist
Description: The proxy and forward plugins are configured simultaneously in the same server block. The newer version of CoreDNS has removed the proxy plugin, and the two plugins cannot coexist.
Solution: Delete the proxy configuration and uniformly use forward:
.:53 {
- proxy . /etc/resolv.conf
+ forward . /etc/resolv.conf
}
5. rewrite Plugin Compatibility Issue (Fatal)
Pre-check Item: Coredns_Plugin_Rewrite_Compatibility_Issues
Description: When an upgrade to version 1.8.4 is performed, incompatible syntax changes were made to the rewrite plugin, requiring adjustments to the current configuration.
Solution: Adjust the configuration according to the rewrite syntax requirements of CoreDNS version 1.8.4. The main changes involve parameter order and keywords. For details, see the CoreDNS rewrite plugin documentation. 6. Upstream DNS Contains Unreachable Addresses (Fatal)
Pre-check Item: Coredns_Plugin_Forward_Bad_Resolve_On_Node
Description: The upstream DNS addresses for CoreDNS contain 127.0.0.53. This address is the local DNS proxy address for some Ubuntu systems. It is unreachable in container environments, which causes DNS forwarding to fail. This address may be configured directly in the upstream section of the Corefile, or it may be sourced from the node's /etc/resolv.conf file (when the Corefile is configured with forward . /etc/resolv.conf).
Solution: In the Corefile, change the upstream DNS to explicitly specify a reachable DNS server address:
.:53 {
- forward . /etc/resolv.conf
+ forward . 183.60.83.19 183.60.82.98
}
Here, 183.60.83.19 and 183.60.82.98 are Tencent Cloud public DNS service addresses, which can also be replaced with your own business DNS servers.
7. Upstream DNS Does Not Support EDNS (Fatal)
Pre-check Item: Coredns_Plugin_Forward_Upstream_Error
Description: The upstream DNS server configured for CoreDNS does not support EDNS (Extension mechanisms for DNS). The newer version of CoreDNS enables EDNS by default. Forwarding queries to an upstream that does not support EDNS causes resolution exceptions.
Solution: In the Corefile, change the upstream DNS to a server that supports EDNS:
.:53 {
- forward . /etc/resolv.conf
+ forward . 183.60.83.19 183.60.82.98
}
8. health Plugin lameduck Timeout Too Long (Warning)
Pre-check Item: Coredns_Plugin_Health_LameDuck_Error
Description: If the lameduck parameter of the health plugin exceeds 30 seconds, it may cause old Pods to terminate slowly during a rolling update, thereby prolonging the upgrade duration.
Solution: Adjust the lameduck parameter to within 30 seconds:
Deployment Configuration
The following pre-check items require changes to the CoreDNS Deployment configuration. Procedure:
kubectl edit deployment coredns -n kube-system
1. Non-standard Container Name (Fatal)
Pre-check Item: Coredns_Container_Not_Standard
Description: The CoreDNS Deployment does not contain a container named coredns. The platform uses this name to locate the container that needs to be updated. If the name does not match, the upgrade cannot be performed.
Solution: Change the name field of the CoreDNS container to coredns.
2. Inconsistent Liveness Probe Configuration (Fatal)
Pre-check Item: Coredns_LivenessProbe_Inconsistent
Description: A mismatch between the LivenessProbe configuration (port, path, or protocol) in the Deployment and the actual listening address of the health plugin in the Corefile causes health checks to fail continuously, resulting in the Pod being restarted repeatedly.
Solution: Modify the LivenessProbe in the Deployment to ensure its port matches that of the health plugin in the Corefile (default: 8080):
livenessProbe:
httpGet:
path: /health
port: 8080
scheme: HTTP
3. Inconsistent Readiness Probe Configuration (Fatal)
Pre-check Item: Coredns_ReadinessProbe_Inconsistent
Description: The ReadinessProbe configuration in the Deployment does not match the actual listening address of the ready plugin in the Corefile.
Solution: Modify the ReadinessProbe in the Deployment to ensure its port matches that of the ready plugin in the Corefile (default: 8181):
readinessProbe:
httpGet:
path: /ready
port: 8181
scheme: HTTP
Cluster Environment and Components
The following pre-check items pertain to cluster-level configurations, such as cluster nodes, the Kubernetes version, or the kube-proxy component.
1. Insufficient Schedulable Nodes (Fatal)
Pre-check Item: CoreDNS_Insufficient_Cluster_Resources
Description: The continuity of the CoreDNS service during the upgrade cannot be guaranteed because the number of schedulable nodes in the cluster is insufficient (less than 2). If the cluster contains super nodes (elastic containers), this item automatically passes.
Solutions:
Add new nodes to the cluster.
Remove the scheduling taint from existing nodes: kubectl uncordon <node-name>
2. Version Incompatibility (Fatal)
Pre-check Item: Coredns_Incompatible_Version
Description: Compatibility constraints exist between the target CoreDNS version and the current cluster environment:
|
CoreDNS ≥ 1.10.1 | Requires Kubernetes version ≥ 1.22.4. |
CoreDNS ≥ 1.11.0 | Docker is not supported as the container runtime; use containerd instead. |
Solutions:
The Kubernetes version is too low: First, upgrade the cluster version.
The container runtime is incompatible: Switch the runtime from Docker to containerd.
3. Non-standard Version (Fatal)
Pre-check Item: Coredns_Non-standard_Version
Description: The current CoreDNS version or the target version is not within the platform-supported upgrade path.
Platform-supported version upgrade path:
|
1.2.2 | 1.6.2 |
1.6.2 | 1.8.4,1.11.1,1.11.1-tke.1 |
1.7.0 | 1.8.4,1.11.1,1.11.1-tke.1 |
1.8.4 | 1.11.1,1.11.1-tke.1 |
1.8.5 | 1.11.1,1.11.1-tke.1 |
1.11.1 | 1.11.1-tke.1 |
When multiple versions are spanned, the platform automatically performs the upgrade in steps (for example, upgrading from 1.2.2 to 1.11.1 will go through the intermediate versions 1.2.2 → 1.6.2 → 1.11.1 sequentially).
Solution: If the current version is not in the list above, first manually adjust the CoreDNS image version to a supported version.
4. iptables Backend Conflict (Fatal)
Pre-check Item: Coredns_KubeProxy_Iptables_Backend_Warning
Description: The coexistence of both iptables-nft and iptables-legacy backend rules on cluster nodes can cause abnormal Service traffic and may affect DNS resolution.
Solution: Upgrade kube-proxy to a version that has fixed this issue:
|
1.14.x | v1.14.3-tke.33-rc2 |
1.16.x | v1.16.3-tke.42-rc2 |
1.18.x | v1.18.4-tke.49-rc1 |
1.20.x | v1.20.6-tke.52 |
1.22.x | v1.22.5-tke.33 |
1.24.x | v1.24.4-tke.24 |
≥ 1.26 | All versions are fixed. |
5. Inconsistent Node DNS Configuration (Warning)
Pre-check Item: Coredns_Plugin_Forward_Upstream_Inconsistent_Each_Node
Description: The nameserver configurations in the /etc/resolv.conf file are inconsistent across cluster nodes. When the Corefile is configured with forward . /etc/resolv.conf, the forwarding behavior of CoreDNS may differ on different nodes.
Solution: Standardize the /etc/resolv.conf configuration across all nodes, or explicitly specify the DNS server address in the Corefile. Alternatively, you can choose to skip this step after confirming the risks.
6. Unreasonable IPVS UDP Timeout Settings (Warning)
Pre-check Item: Coredns_KubeProxy_IPVS_Timeout_Warning
Description: When kube-proxy uses IPVS mode, if the --ipvs-udp-timeout parameter is not set or exceeds 10 seconds, it may cause intermittent DNS resolution timeouts.
Solution: Configure the startup parameter --ipvs-udp-timeout=10s for kube-proxy, or choose to skip after confirming the risks.
Pre-check Items That Do Not Require Handling
The following pre-check items typically do not require user action:
|
Coredns_Cluster_Status
| The cluster status is abnormal (not Running). Wait for the cluster to recover and then retry. |
Coredns_Cluster_APIServer
| The platform cannot connect to the cluster API Server. This is usually a platform-side issue. If the problem persists, submit a ticket. |
CoreDNS_Already_Update
| The current version is consistent with the target version. No upgrade is required. |
Coredns_KubeProxy_Binary_Warning
| kube-proxy is deployed as a binary. The platform cannot automatically check its configuration. Manual verification is recommended (Warning, can be skipped). |
Coredns_Plugin_Forward_Upstream_Not_Subset_Metrics
| Unexpected upstream DNS addresses appear in CoreDNS monitoring metrics. This is usually a historical residue and generally has no impact (Warning, can be skipped). |