Annotation

Last updated: 2021-06-25 16:24:13

    Workload Template Annotation Description

    You can define template annotation in a YAML file to implement capabilities such as binding security groups and allocating resources for Pods. For more information about the configuration method, please see the following table.

    Note:

    • If no security group is specified, a Pod is bound with the default security group in the same region by default. Ensure that the network policy of the default security group does not affect the Pod.
    • To allocate GPU resources, you must specify eks.tke.cloud.tencent.com/gpu-type.
    • Except eks.tke.cloud.tencent.com/gpu-type, the other four annotations related to resource allocation in the following table are optional. If you specify them, ensure that they are correct.
    • To allocate CPU resources, you must specify both cpu and mem annotations and make sure that their values meet the CPU specifications in Resource Specifications. In addition, you can select Intel or AMD CPUs to allocate by specifying cpu-type. AMD CPUs are more cost-effective. For more information, see Product Pricing.
    • To allocate GPU resources, you must specify the cpu, mem, gpu-type, and gpu-count annotations and ensure that their values meet the GPU specifications in Resource Specifications.
    Annotation Key Annotation Value and Description Required
    eks.tke.cloud.tencent.com/security-group-id Default security group bound with a workload. Specify the security group ID.
    • Multiple security group IDs can be specified and separated by a comma (,), such as sg-id1,sg-id2.
    • Network policies take effect based on the sequence of security groups.
    • Please note that a single security group can be associated with only 2,000 computing instances, such as CVM instances and Elastic Kubernetes Service (EKS) Pods. For more information, see Security Group Limits.
    No. If you do not specify it, the default security group in the same region bound to the workload is associated by default.
    If you specify it, ensure that the security group ID already exists in the region where the workload resides.
    eks.tke.cloud.tencent.com/cpu Number of CPU cores required by a Pod. For more information, see Resource Specifications. The unit is core by default. No. If you specify it, ensure that the specifications are supported and specify the cpu and mem parameters.
    eks.tke.cloud.tencent.com/mem Memory required by a Pod. For more information, see Resource Specifications. The unit must be included in the value, for example, 512 MiB, 0.5 GiB, or 1 GiB. No. If you specify it, ensure that the specifications are supported and specify the cpu and mem parameters.
    eks.tke.cloud.tencent.com/cpu-type Model of the CPU resources required by a Pod. Currently, the supported models include:
    • intel
    • amd
    • You can specify the model by priority. For example, `amd,intel` indicates AMD resource Pods will be created first. If the AMD resources in the selected region are insufficient, Intel resource Pods will be created.
    For specific configurations supported by each model, please see Resource Specifications.
    No. If you do not specify it, the CPU type is not forcibly specified by default. The system will match the most suitable specifications according to Specifying Resource Specifications. If the matched specifications are supported by both Intel and AMD, Intel CPUs are preferred.
    eks.tke.cloud.tencent.com/gpu-type Model of the GPU resources required by a Pod. Currently, the supported models include:
    • V100
    • 1/4*T4
    • 1/2*T4
    • T4
    • You can specify the model by priority. For example, “T4,V100” indicates T4 resource Pods will be created first. If the T4 resources in the selected region are insufficient, V100 resource Pods will be created.
    For specific configurations supported by each model, please see Resource Specifications.
    If GPUs are required, this option is required. When specifying it, ensure that the GPU model is supported. Otherwise, an error will be reported.
    eks.tke.cloud.tencent.com/gpu-count Number of GPUs required by a Pod. For more information, please see Resource Specifications. The unit is card by default. No. If you specify it, ensure that the specification is supported.
    eks.tke.cloud.tencent.com/retain-ip The static IP of a Pod. Enter the value "true" to enable this feature. If a Pod with the static IP enabled is terminated, its IP will be retained 24 hours by default. If the Pod is rebuilt within 24 hours after termination, its IP can still be used. Otherwise, its IP may be occupied by other Pod. No
    eks.tke.cloud.tencent.com/retain-ip-hours Modifies the default retention duration of the Pod’s static IP. Enter a number. Unit: hour. Default value: 24 hours. The IP can be retained up to one year. No
    eks.tke.cloud.tencent.com/role-name Associates a Pod with a CAM role. Please specify CAM role name as the value. In this way, the Pod can obtain the permission policies of the associated CAM role to facilitate cloud resource operations such as purchasing resources and reading from or writing to storage. No. If you specify it, please make sure the specified CAM role exists.
    eks.tke.cloud.tencent.com/monitor-port Opens a port for monitoring data for a Pod, so as to facilitate collection by PROM instance and other components. No. If you do not specify it, the port will default to 9100.
    eks.tke.cloud.tencent.com/custom-metrics-url Sets a custom monitoring metric pull address for a Pod. The monitoring data opened at this address will be automatically read and reported by the monitoring component. No. If you specify it, please ensure that the opened data protocol can be recognized by the monitoring system, such as the Prometheus protocol and cloud monitoring data protocol.

    Sample

    The following example shows the complete GPU specifications of the security group bound to a Pod.

    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      generation: 1
      labels:
        k8s-app: nginx
        qcloud-app: nginx
      name: nginx
      namespace: default
    spec:
      progressDeadlineSeconds: 600
      replicas: 1
      revisionHistoryLimit: 10
      selector:
        matchLabels:
          k8s-app: nginx
          qcloud-app: nginx
      strategy:
        rollingUpdate:
          maxSurge: 1
          maxUnavailable: 0
        type: RollingUpdate
      template:
        metadata:
          annotations:
            eks.tke.cloud.tencent.com/cpu: "2"
            eks.tke.cloud.tencent.com/gpu-count: "1"
            eks.tke.cloud.tencent.com/gpu-type: 1/4*V100
            eks.tke.cloud.tencent.com/mem: 10Gi
            eks.tke.cloud.tencent.com/security-group-id: "sg-dxxxxxx5,sg-zxxxxxxu"
            eks.tke.cloud.tencent.com/role-name: "cam-role-name"
            eks.tke.cloud.tencent.com/monitor-port: "9123"
            eks.tke.cloud.tencent.com/custom-metrics-url: "http://localhost:8080/metrics"
          creationTimestamp: null
          labels:
            k8s-app: nginx
            qcloud-app: nginx
        spec:
          containers:
          - image: nginx:latest
            imagePullPolicy: Always
            name: nginx
            resources:
              limits:
                cpu: "1"
                memory: 2Gi
                nvidia.com/gpu: "1"
              requests:
                cpu: "1"
                memory: 2Gi
                nvidia.com/gpu: "1"
            terminationMessagePath: /dev/termination-log
            terminationMessagePolicy: File
          dnsPolicy: ClusterFirst
          imagePullSecrets:
          - name: qcloudregistrykey
          restartPolicy: Always
          schedulerName: default-scheduler
          securityContext: {}
          terminationGracePeriodSeconds: 30
    

    Virtual Node Annotation Description

    EKS supports the virtual nodes. You can specify annotations in a YAML file to implement capabilities such as custom DNS, as shown below:

    Annotation Key Annotation Value and Description Required
    eks.tke.cloud.tencent.com/resolv-conf Queries the list of IP addresses for the DNS server while resolving the domain name, for example nameserver 8.8.8.8.
    You can use kubectl edit node eklet-subnet-xxxx to add this annotation.
    After the modification, the Pods scheduled to this virtual node will adopt this DNS configuration by default.
    No

    Sample

    The example of a custom DNS configuration for a virtual node is as follows:

    apiVersion: v1
    kind: Node
    metadata:
      annotations:
        eks.tke.cloud.tencent.com/resolv-conf:|
          nameserver 4.4.4.4
          nameserver 8.8.8.8
    
    

    Service Annotation Description

    EKS allows you to use existing CLBs to create services accessed through the public or private network. If you want to provide idle CLBs to created services or use the same CLB in a cluster, you can add annotations.

    Annotation Key Annotation Value and Description Required
    service.kubernetes.io/tke-existed-lbid The Service is created with the existing CLB. Specify the ID of the CLB instance you want to use as the value. No. If you specify it, ensure that the specified CLB instance ID exists.
    service.kubernetes.io/qcloud-share-existed-lb By default, multiple Services cannot share the same CLB instance. If you hope that a Service uses the CLB occupied by other Services, please add this annotation and specify the value as "true". No. If you do not specify it, a CLB instance cannot be reused by default.

    The elastic cluster also supports the same expansion protocol as the TKE cluster. For more information, see Service Extension Protocol.

    Note:

    • Ensure that your EKS does not share the same CLB with the CVM.
    • When the existing CLBs are used:
      • Only CLBs created through the CLB console can be used. You cannot reuse CLBs automatically created by TKE.
      • Ports of Services that share the same existing CLB cannot be the same.
      • Cross-cluster Services cannot share the same CLB.

    Sample

    apiVersion: v1
    kind: Service
    metadata:
      annotations:
        service.kubernetes.io/tke-existed-lbid: lb-pxxxxxxq
        service.kubernetes.io/qcloud-share-existed-lb: true
      name: servicename
      namespace: default
    spec:
      externalTrafficPolicy: Cluster
      ports:
      - name: tcp-80-80
        nodePort: 31728
        port: 80
        protocol: TCP
        targetPort: 80
      sessionAffinity: None
      type: LoadBalancer