tencent cloud

Feedback

Non-static IP Address Mode Instructions

Last updated: 2021-12-10 10:12:38

    Overview

    The non-static IP address mode is suitable for the scenarios where the service does not rely on static IPs of the container. For example, stateless services and stateless offline applications that can deploy multiple copies.

    Features and Limitations

    • The node can maintain available ENI/IP pool, so that the Pod can be rebuilt quickly in a massive way.
    • It supports the pre-binding policy, so that the Pod can be scaled out quickly.
    • It supports auto scaling of ENIs/IPs to avoid waste of IPs and improve their utilization rate.
    • 0 cannot be set for the pre-binding. That means fully on-demand allocation is not supported for now, and waste of IPs may occur if there are too many nodes.

    IP Address Management Principle

    TKE maintains an auto-scaling exclusive ENI/IP pool on each node. The number of bound exclusive ENIs/IPs ranges from the number of Pods + the minimum number of pre-bound ENIs/IPs to the number of Pods + the maximum number of pre-bound ENIs/IPs.

    • When the number of bound ENIs/IPs is less than the number of Pods + the minimum number of pre-bound ENIs/IPs, new exclusive ENIs/IPs will be bound to make the number of bound ENIs/IPs = the number of Pods + the minimum number of pre-bound ENIs/IPs.
    • When the number of bound ENIs/IPs is larger than the number of Pods + the maximum number of pre-bound ENIs/IPs, an ENI/IP will be released about every 2 minutes until the number of bound ENIs/IPs = the number of Pods + the maximum number of pre-bound ENIs/IPs.
    • When the maximum number of bindable ENIs/IPs is less than the number of bound ENIs/IPs, the unnecessary idle exclusive ENIs/IPs will be released directly to make **the number of bound ENIs/IPs equal to the maximum number of bindable ENIs/IPs.

    Directions

    You can enable the non-static IP address mode by the following method:

    • Select non-static IP address of VPC-CNI mode when creating a cluster. That means you should not check Enable Support for Static Pod IP.

    Supporting quick release

    The ENI/IP pool managed in non-static IP address mode applies the policy of slow release by default. Only one unnecessary idle ENI/IP is released every two minutes. If you want to utilize IPs more efficiently, you need to enable "quick release". In this mode, the ENI/IP pool is checked every two minutes to release unnecessary idle ENIs/IPs until the number of idle ENIs/IPs is equal to the maximum number of pre-bound ENIs/IPs.

    Method for enabling

    • Run the command kubectl edit ds tke-eni-agent -n kube-system to modify the existing tke-eni-agent daemonset.
    • Add the following launch parameter to spec.template.spec.containers[0].args to enable "quick release". The tke-eni-agent will update and make the feature take effect on a rolling basis after the modification.
      - --enable-quick-release
      

    Specifying the number of pre-bound ENIs/IPs on a node

    You can specify the number of pre-bound ENIs/IPs by modifying the annotation of CRD NEC corresponding to the node.

    # Specifying the minimum number of pre-bound ENIs/IPs in shared ENI mode
    "tke.cloud.tencent.com/route-eni-ip-min-warm-target"
    # Specifying the maximum number of pre-bound ENIs/IPs in shared ENI mode
    "tke.cloud.tencent.com/route-eni-ip-max-warm-target"
    # Specifying the minimum number of pre-bound ENIs/IPs in exclusive ENI mode
    "tke.cloud.tencent.com/direct-eni-min-warm-target"
    # Specifying the maximum number of pre-bound ENIs/IPs in exclusive ENI mode
    "tke.cloud.tencent.com/direct-eni-max-warm-target"
    

    How to modify:

    # Sample: modify the minimum number of pre-bound IPs on <nodeName> to 1
    kubectl annotate nec <nodeName> "tke.cloud.tencent.com/route-eni-ip-min-warm-target"="1" --overwrite
    # Sample: modify the maximum number of pre-bound IPs on <nodeName> to 3
    kubectl annotate nec <nodeName> "tke.cloud.tencent.com/route-eni-ip-max-warm-target"="3" --overwrite
    
    • Dynamic check for pre-binding is triggered immediately after the modification. If the number of pre-bound ENIs/IPs is insufficient, ENIs/IPs will be bound until the number meets your requirement. Otherwise, the ENIs/IPs will be unbound.
    • The two annotations must exist at the same time when you perform the modification, and the condition of 0 &lt;= the minimum number of pre-bound ENIs/IPs &lt;= the maximum number of pre-bound ENIs/IPs must be met. Otherwise, the modification will be failed.

    Specifying the maximum number of bindable ENIs/IPs on a node

    You can specify the maximum number of bindable ENIs/IPs on a node by modifying the annotation of CRD nec corresponding to the node. You can use the following annotations to specify the maximum number of bindable ENIs and the maximum number of bindable IPs on a single ENI.

    # Specifying the maximum number of bindable ENIs in shared ENI mode
    kubectl annotate nec <nodeName> "tke.cloud.tencent.com/route-eni-max-attach"="1" --overwrite
    # Specifying the number of bindable IPs on a single ENI in shared ENI mode
    kubectl annotate nec <nodeName> "tke.cloud.tencent.com/max-ip-per-route-eni"="9" --overwrite
    # Specifying the maximum number of bindable exclusive ENIs in exclusive ENI mode
    kubectl annotate nec <nodeName> "tke.cloud.tencent.com/direct-eni-max-attach"="5" --overwrite
    

    You must ensure that the value you entered for modification is not less than the number of ENIs/IPs currently in use on the node. Otherwise, the modification will be failed.
    Dynamic check for pre-binding is triggered immediately after the modification. If the number of bound ENIs/IPs > the maximum number of bindable ENIs/IPs, the ENIs/IPs will be unbound until the number of bound ENIs/IPs = the maximum number of bindable ENIs/IPs.

    Specifying the default number of pre-bound ENIs/IPs

    • Run the command kubectl edit deploy tke-eni-ipamd -n kube-system to modify the existing tke-eni-ipamd deployment.
    • Add the following launch parameter to spec.template.spec.containers[0].args to modify the default number of pre-bound ENIs/IPs. The ipamd will restart and take effect automatically after the modification. The default number only affects the added nodes.
      # The default number of minimum pre-bound ENIs/IPs in shared ENI mode. Default value: 5
    • --ip-min-warm-target=3

      The default number of maximum pre-bound ENIs/IPs in shared ENI mode. Default value: 5

    • --ip-max-warm-target=3

      The default number of minimum pre-bound ENIs/IPs in exclusive ENI mode. Default value: 1

    • --eni-min-warm-target=3

      The default number of maximum pre-bound ENIs/IPs in exclusive ENI mode. Default value: 1

    • --eni-max-warm-target=3
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support