tencent cloud

Feedback

Static Mounting of CFS-Turbo for TKE Clusters

Last updated: 2022-06-22 11:32:32

    Overview

    You can mount a CFS Turbo storage for a TKE cluster by installing a kubernetes-csi-tencentloud add-on. This add-on is used to mount the Tencent Cloud CFS Turbo file system to a workload based on a private protocol. Currently, only static configuration is supported. For more information about CFS storage types, see Storage Types and Performance.

    Prerequisites

    You have created a TKE cluster or created a Kubernetes cluster on Tencent Cloud, and the cluster version is 1.14 or later.

    Directions

    Creating a file system

    Create a CFS Turbo file system. For details, see Creating File Systems and Mount Targets.

    Note:

    After the file system is created, you need to associate the cluster network (vpc-xx) with the CCN instance of the file system. You can check it in the information about the file system mount target.

    Deploying a RBAC policy

    If you want to mount a CFS Turbo volume, you need to run the kubectl apply -f csi-node-rbac.yaml command to deploy a RBAC policy in the cluster. The following csi-node-rbac.yaml code is for your reference:

    apiVersion: v1
    kind: ServiceAccount
    metadata:
    name: cfsturbo-csi-node-sa
    namespace: kube-system
    ---
    kind: ClusterRole
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
    name: cfsturbo-csi-node-role
    rules:
    - apiGroups: [""]
      resources: ["persistentvolumes", "endpoints", "configmaps"]
      verbs: ["get", "list", "watch", "create", "delete", "update"]
    - apiGroups: [""]
      resources: ["persistentvolumeclaims", "nodes"]
      verbs: ["get", "list", "watch", "update"]
    - apiGroups: [""]
      resources: ["events"]
      verbs: ["get", "list", "watch", "create", "update", "patch"]
    - apiGroups: [""]
      resources: ["secrets", "namespaces"]
      verbs: ["get", "list"]
    - apiGroups: [""]
      resources: ["nodes", "pods"]
      verbs: ["get", "list", "watch", "update"]
    - apiGroups: ["storage.k8s.io"]
      resources: ["volumeattachments", "volumeattachments"]
      verbs: ["get", "list", "watch", "update", "patch"]
    - apiGroups: ["storage.k8s.io"]
      resources: ["storageclasses"]
      verbs: ["get", "list", "watch"]
    ---
    kind: ClusterRoleBinding
    apiVersion: rbac.authorization.k8s.io/v1
    metadata:
    name: cfsturbo-csi-node-rolebinding
    subjects:
    - kind: ServiceAccount
      name: cfsturbo-csi-node-sa
      namespace: kube-system
    roleRef:
    kind: ClusterRole
    name: cfsturbo-csi-node-role
    apiGroup: rbac.authorization.k8s.io
    

    Deploying a Node Plugin

    1. Run the kubectl apply -f csidriver.yaml command. The following csidriver.yaml code is for your reference:
      apiVersion: storage.k8s.io/v1beta1
      kind: CSIDriver
      metadata:
      name: com.tencent.cloud.csi.cfsturbo
      spec:
      attachRequired: false
      podInfoOnMount: false
      
    1. Run the kubectl apply -f csi-node.yaml commad. The following csi-node.yaml code is for your reference:
      # This YAML file contains driver-registrar & csi driver nodeplugin API objects
      # that are necessary to run CSI nodeplugin for cfsturbo
      kind: DaemonSet
      apiVersion: apps/v1
      metadata:
      name: cfsturbo-csi-node
      namespace: kube-system
      spec:
      selector:
      matchLabels:
        app: cfsturbo-csi-node
      template:
      metadata:
        labels:
          app: cfsturbo-csi-node
      spec:
        serviceAccount: cfsturbo-csi-node-sa
        hostNetwork: true
        containers:
          - name: driver-registrar
            image: ccr.ccs.tencentyun.com/tkeimages/csi-node-driver-registrar:v1.2.0
            lifecycle:
              preStop:
                exec:
                  command: ["/bin/sh", "-c", "rm -rf /registration/com.tencent.cloud.csi.cfsturbo /registration/com.tencent.cloud.csi.cfsturbo-reg.sock"]
            args:
              - "--v=5"
              - "--csi-address=/plugin/csi.sock"
              - "--kubelet-registration-path=/var/lib/kubelet/plugins/com.tencent.cloud.csi.cfsturbo/csi.sock"
            env:
              - name: KUBE_NODE_NAME
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
            volumeMounts:
              - name: plugin-dir
                mountPath: /plugin
              - name: registration-dir
                mountPath: /registration
          - name: cfsturbo
            securityContext:
              privileged: true
              capabilities:
                add: ["SYS_ADMIN"]
              allowPrivilegeEscalation: true
            image: ccr.ccs.tencentyun.com/tkeimages/csi-tencentcloud-cfsturbo:v1.2.2
            args :
              - "--nodeID=$(NODE_ID)"
              - "--endpoint=$(CSI_ENDPOINT)"
            env:
              - name: NODE_ID
                valueFrom:
                  fieldRef:
                    fieldPath: spec.nodeName
              - name: CSI_ENDPOINT
                value: unix://plugin/csi.sock
            imagePullPolicy: "IfNotPresent"
            volumeMounts:
              - name: plugin-dir
                mountPath: /plugin
              - name: pods-mount-dir
                mountPath: /var/lib/kubelet/pods
                mountPropagation: "Bidirectional"
              - name: global-mount-dir
                mountPath: /etc/cfsturbo/global
                mountPropagation: "Bidirectional"
        volumes:
          - name: plugin-dir
            hostPath:
              path: /var/lib/kubelet/plugins/com.tencent.cloud.csi.cfsturbo
              type: DirectoryOrCreate
          - name: pods-mount-dir
            hostPath:
              path: /var/lib/kubelet/pods
              type: Directory
          - name: registration-dir
            hostPath:
              path: /var/lib/kubelet/plugins_registry
              type: Directory
          - name: global-mount-dir
            hostPath:
              path: /etc/cfsturbo/global
              type: DirectoryOrCreate
      

    Using a CFS Turbo volume

    1. Create a CFS Turbo file system. For more information, see Creating a File System.
    2. Use the following template to create a PV of CFS Turbo type.
      apiVersion: v1
      kind: PersistentVolume
      metadata:
      name: pv-cfsturbo
      spec:
      accessModes:
      - ReadWriteMany
      capacity:
      storage: 10Gi
      csi:
      driver: com.tencent.cloud.csi.cfsturbo
      # volumeHandle in PV must be unique, use pv name is better
      volumeHandle: pv-cfsturbo
      volumeAttributes: 
      # cfs turbo server ip
      host: 10.0.0.116
      # cfs turbo fsid (not cfs id)
      fsid: xxxxxxxx
      # cfs turbo rootdir
      rootdir: /cfs
      # cfs turbo subPath
      path: /
      proto: lustre
      storageClassName: ""
      

    Parameter description:

    • metadata.name: The name of the created PV.
    • spec.csi.volumeHandle: It must be consistent with the PV name.
    • spec.csi.volumeAttributes.host: The IP address of the file system. You can check it in the information about file system mount target.
    • spec.csi.volumeAttributes.fsid: The fsid of the file system (not the file system ID). You can check it in the file system mount target information. It is the string after "tcp0:/" and before "/cfs" in the mounting command, as shown in the following figure.
    • spec.csi.volumeAttributes.rootdir: The root directory of the file system. “/cfs” is entered if it is left empty (the general mounting performance is enhanced if mounting to “/cfs”). If you want to specify a root directory for mounting, you must ensure that the root directory exists in the file system.
    • spec.csi.volumeAttributes.path: The subdirectory of the file system. “/” is entered if it is left empty. If you want to specify a subdirectory for mounting, you must ensure that the subdirectory exists in rootdir of the file system. The directory accessed by the container is the rootdir+path directory of the file system (defaults to “/cfs/” directory).
    • spec.csi.volumeAttributes.proto: The default protocol for mounting the file system.
      Note:

      You need to install a Client in the cluster node according to the version of operating system kernel before using lustre protocol to mount a CFS Turbo volume. For details, see Using CFS Turbo on Linux Clients.

    1. Use the following template to create a PVC that binds a PV.
      apiVersion: v1
      kind: PersistentVolumeClaim
      metadata:
      name: pvc-cfsturbo
      spec:
      storageClassName: ""
      volumeName: pv-cfsturbo
      accessModes:
      - ReadWriteMany
      resources:
      requests:
      storage: 10Gi
      

    Parameter description:

    • metadata.name: The name of the created PVC.
    • spec.volumeName: This need to be consistent with the name of PV created in the previous step.
    1. Use the following template to create a Pod that mounts a PVC.
      apiVersion: v1
      kind: Pod
      metadata:
      name: nginx 
      spec:
      containers:
      - image: ccr.ccs.tencentyun.com/qcloud/nginx:1.9
      imagePullPolicy: Always
      name: nginx
      ports:
      - containerPort: 80
      protocol: TCP
      volumeMounts:
      - mountPath: /var/www
        name: data
      volumes:
      - name: data
      persistentVolumeClaim:
      claimName: pvc-cfsturbo
      
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support