tencent cloud

PrevNext

masukan

cari berdasarkan kata kunci

Recent Pages

Dokumentasi
Cloud Object Storage
    Dokumentasi

    Global Client Deployment

    Terakhir diperbarui:2024-03-25 16:04:01
      In Fluid, remote files defined in the 'Dataset' resource object are schedulable, which means you can manage the location for caching remote files on a Kubernetes cluster as you manage your pods. The pod that performs the computing can access the data files through the Fuse client.
      The Fuse client supports two modes:
      global is false: In this mode, affinity is enforced between Fuse clients and cache data, and the number of Fuse clients is equal to the number of Runtime replicas. This is the default mode and does not need to be explicitly declared. The advantage of this mode is that it takes advantage of data affinity, but the deployment of Fuse clients becomes more rigid.
      global is true: In this mode, Fuse clients can be deployed globally in a Kubernetes cluster. There is no mandatory affinity between data and Fuse clients. In this case, the number of Fuse clients may far exceed the number of Runtime replicas. It is recommended that nodeSelector be used to specify the deployment scope of Fuse clients.

      Prerequisites

      Before running the sample code provided in this document, install Fluid by referring to Installation. Be sure to run the helm command to add the parameter --set webhook.enable=true to enable webhook and check that all the components used by Fluid are running properly.
      $ kubectl get pod -n fluid-system
      goosefsruntime-controller-5b64fdbbb-84pc6   1/1     Running   0          8h
      csi-nodeplugin-fluid-fwgjh                  2/2     Running   0          8h
      csi-nodeplugin-fluid-ll8bq                  2/2     Running   0          8h
      dataset-controller-5b7848dbbb-n44dj         1/1     Running   0          8h
      Normally, you shall see a pod named dataset-controller, a pod named goosefsruntime-controller, and multiple pods named csi-nodeplugin. The number of csi-nodeplugin pods depends on the number of nodes in your Kubernetes cluster.

      Setting Up an Environment

      $ mkdir <any-path>/fuse-global-deployment
      $ cd <any-path>/fuse-global-deployment

      Demo Run Example

      Example 1: Set global to true

      Check all nodes in your cluster
      $ kubectl get nodes
      NAME            STATUS   ROLES    AGE     VERSION
      192.168.1.146   Ready    <none>   7d14h   v1.18.4-tke.13
      192.168.1.147   Ready    <none>   7d14h   v1.18.4-tke.13
      Label one of the nodes
      $ kubectl label nodes 192.168.1.146 cache-node=true
      In the following steps, we will use NodeSelector to manage where to store data, and therefore we add a label to a desired node.
      Check all nodes again
      $ kubectl get node -L cache-node
      NAME         STATUS   ROLES    AGE     VERSION            cache-node
      192.168.1.146   Ready    <none>   7d14h   v1.18.4-tke.13   true
      192.168.1.147   Ready    <none>   7d14h   v1.18.4-tke.13
      Currently, among all the 2 nodes, only one node holds the cache-node=true label, indicating that data will only be cached on this node.
      Check the Dataset resource object to be created
      apiVersion: data.fluid.io/v1alpha1
      kind: Dataset
      metadata:
        name: hbase
      spec:
        mounts:
          - mountPoint: https://mirrors.tuna.tsinghua.edu.cn/apache/hbase/stable/
            name: hbase
        nodeAffinity:
          required:
            nodeSelectorTerms:
              - matchExpressions:
                  - key: cache-node
                    operator: In
                    values:
                      - "true"
      Note:
      To facilitate testing, mountPoint is set to Web UFS in this example. If you want to mount COS, see Mounting COS (COSN) to GooseFS.
      In the spec attribute of the Dataset resource object, we defined the nodeSelectorTerms sub-attribute to specify that data caches must be placed on the node with the cache-node=true label.
      Create the Dataset resource object
      $ kubectl create -f dataset.yaml
      dataset.data.fluid.io/hbase created
      Check the GooseFSRuntime resource object to be created
      apiVersion: data.fluid.io/v1alpha1
      kind: GooseFSRuntime
      metadata:
        name: hbase
      spec:
        replicas: 1
        tieredstore:
          levels:
            - mediumtype: SSD
              path: /mnt/disk1/
              quota: 2G
              high: "0.8"
              low: "0.7"
        fuse:
          global: true
      The above snippet of the configuration file contains a lot of GooseFS related configuration, and Fluid will start a GooseFS instance based on the configuration. Among the configuration, the spec.replicas attribute is set to 1, indicating that Fluid will start a GooseFS instance containing 1 GooseFS master and 1 GooseFS worker. It is also worth noting that Fuse contains `global: true', which means that Fuse can be deployed globally, independent of the location of the data cache.
      Create the GooseFSRuntime resource object and check its status
      $ kubectl create -f runtime.yaml
      goosefsruntime.data.fluid.io/hbase created
      
      
      $ kubectl get po -owide
      NAME                 READY   STATUS    RESTARTS   AGE     IP              NODE                       NOMINATED NODE   READINESS GATES
      hbase-fuse-gfq7z     1/1     Running   0          3m47s   192.168.1.147   192.168.1.147   <none>           <none>
      hbase-fuse-lmk5p     1/1     Running   0          3m47s   192.168.1.146   192.168.1.146   <none>           <none>
      hbase-master-0       2/2     Running   0          3m47s   192.168.1.147   192.168.1.147   <none>           <none>
      hbase-worker-hvbp2   2/2     Running   0          3m1s    192.168.1.146   192.168.1.146   <none>           <none>
      As shown above, one GooseFS worker is started successfully and runs on the node with the cache-node=true label. There are two GooseFS Fuse clients, running on all sub-nodes.
      Check the status of the GooseFSRuntime object
      $ kubectl get goosefsruntime hbase -o wide
      NAME    READY MASTERS   DESIRED MASTERS   MASTER PHASE   READY WORKERS   DESIRED WORKERS   WORKER PHASE   READY FUSES   DESIRED FUSES   FUSE PHASE   AGE
      hbase   1               1                 Ready          1               1                 Ready          2             2               Ready        12m
      As shown above, there is one GooseFS worker and two GooseFS Fuse clients.
      Delete GooseFSRuntime
      kubectl delete goosefsruntime hbase

      Example 2: Set global to true and use nodeSelector to configure the Fuse client

      In this example, we use nodeSelector to configure the Fuse client to place it to a certain node in the cluster. Now that we have selected the 192.168.1.146 node as the cache node, for comparison, we select the 192.168.1.147 node to run GooseFS Fuse.
      apiVersion: data.fluid.io/v1alpha1
      kind: GooseFSRuntime
      metadata:
        name: hbase
      spec:
        replicas: 1
        tieredstore:
          levels:
            - mediumtype: SSD
              path: /mnt/disk1/
              quota: 2G
              high: "0.8"
              low: "0.7"
        fuse:
          global: true
          nodeSelector:
            kubernetes.io/hostname: 192.168.1.147
      In this configuration file snippet, compared with runtime.yaml, nodeSelector is added and points to the 192.168.1.147 node while Fuse contains 'global: true'.
      Create the GooseFSRuntime resource object and check its status
      $ kubectl create -f runtime-node-selector.yaml
      goosefsruntime.data.fluid.io/hbase created
      
      
      $ kubectl get po -owide
      NAME                 READY   STATUS    RESTARTS   AGE   IP              NODE                       NOMINATED NODE   READINESS GATES
      hbase-fuse-xzbww     1/1     Running   0          1h   192.168.1.147   192.168.1.147   <none>           <none>
      hbase-master-0       2/2     Running   0          1h   192.168.1.147   192.168.1.147   <none>           <none>
      hbase-worker-vdxd5   2/2     Running   0          1h   192.168.1.146   192.168.1.146   <none>           <none>
      As shown above, one GooseFS worker is started successfully and runs on the node with the cache-node=true label. There is one GooseFS Fuse client, running on the sub-node 192.168.1.147.
      Check the status of the GooseFSRuntime object
      $ kubectl get goosefsruntimes.data.fluid.io -owide
      NAME    READY MASTERS   DESIRED MASTERS   MASTER PHASE   READY WORKERS   DESIRED WORKERS   WORKER PHASE   READY FUSES   DESIRED FUSES   FUSE PHASE   AGE
      hbase   1               1                 Ready          1               1                 Ready          1             1               Ready        1h
      As shown above, there is one GooseFS worker and one GooseFS Fuse client. This is because GooseFSRuntime specifies nodeSelector and only one node meets the specification.
      This example shows that Fluid supports separate scheduling policies for Fuse clients, providing users with more flexible Fuse client scheduling policies.

      Cleaning Up the Environment

      $ kubectl delete -f .
      
      $ kubectl label node 192.168.1.146 cache-node-
      Hubungi Kami

      Hubungi tim penjualan atau penasihat bisnis kami untuk membantu bisnis Anda.

      Hubungi Kami
      Dukungan Teknis

      Buka tiket jika Anda mencari bantuan lebih lanjut. Tiket kami tersedia 7x24.

      Kirim Tiket
      Dukungan Telepon 7x24
      Copyright © 2013-2024 Tencent Cloud. All Rights Reserved.
      Kebijakan PrivasiLegalKebijakan Cookie