tencent cloud

Feedback

Connecting Self-built Kubernetes Cluster to CLS

Last updated: 2021-12-30 14:31:39

    Overview

    CLS supports log collection for self-built Kubernetes clusters. Before performing log collection on a self-built Kubernetes cluster, you need to use a custom resource definition (CRD) to define log collection configuration (LogConfig), and deploy Log-Provisioner, Log-Agent, and LogListener on the cluster. If you are a Tencent Kubernetes Engine (TKE) user, you can quickly access and use the CLS service by referring to Enabling log collection.

    Prerequisites

    • You have created a cluster of Kubernetes 1.10 or above.
    • You have enabled CLS, created a logset and a log topic, and obtained the log topic ID (topicId).
      For configuration details, please see Creating a log topic.
    • You have obtained the domain name (CLS_HOST) of the region of your log topic.
      For details of the CLS domain name list, see Available Regions.
    • You have obtained the API key ID (TmpSecretId) and API key (TmpSecretKey) required for CLS authentication.
      To obtain the API key and API key ID, go to Manage API Key.

    Kubernetes Log Collection Principles

    Log collection on a Kubernetes cluster requires the following:

    • LogConfig: CRD of log collection configuration, which defines where logs are collected, how they are parsed, and to which CLS log topic they are shipped after being parsed.
    • Log-Provisioner: component used to synchronize the log collection configuration defined in LogConfig to CLS.
    • Log-Agent: component used to listen for changes in LogConfig and containers on nodes and dynamically calculate the actual positions of log files in containers on node hosts.
    • LogListener: component used to collect log file content from node hosts, parses it, and uploads it to CLS.

    Directions

    Directions

    Step 1. Define the LogConfig resource type

    Define the LogConfig resource type using a Kubernetes CRD.
    Run the wget command to download the CRD.yaml declaration file and run the kubectl command to define the LogConfig resource type, using the master node path /usr/local/ as an example.

    wget https://mirrors.tencent.com/install/cls/k8s/CRD.yaml
    kubectl create -f /usr/local/CRD.yaml
    

    Step 2. Define the LogConfig object

    Create a LogConfig object to define log collection configuration. Run the wget command to download the LogConfig.yaml declaration file, using the master node path /usr/local/ as an example.

    wget https://mirrors.tencent.com/install/cls/k8s/LogConfig.yaml
    

    The LogConfig.yaml declaration file consists of the following two parts:

    • clsDetail: defines the log parsing format and the target log topic ID (topicId).
    • inputDetail: defines the log source from which logs are collected.
    Note:

    During configuration, change the topicId item in clsDetail to the ID of the log topic that you created.

    Log parsing format

    In "full text in a single line" mode, a line is a full log. When CLS collects logs, it uses the line break \n to mark the end of a log. For easier structural management, a default key value \_\_CONTENT\_\_ is given to each log, but the log data itself will no longer be structured, nor will the log field be extracted. The time attribute of a log is determined by the collection time.

    Assume that the raw data of a log is as follows:

    Tue Jan 22 12:08:15 CST 2019 Installed: libjpeg-turbo-static-1.2.90-6.el7.x86_64
    

    A sample of LogConfig configuration is as follows:

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     clsDetail:
       topicId: xxxxxx-xx-xx-xx-xxxxxxxx
       # Single-line log
       logType: minimalist_log
    

    The data collected to CLS is as follows:

    __CONTENT__:Tue Jan 22 12:08:15 CST 2019 Installed: libjpeg-turbo-static-1.2.90-6.el7.x86_64
    

    Log source

    CLS supports the following cluster log sources:

    Sample 1: collecting the standard output of all containers in the default namespace

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     inputDetail:
       type: container_stdout
       containerStdout:
         namespace: default
         allContainers: true
    ...
    

    Sample 2: collecting the container standard output in the Pod that belongs to ingress-gateway deployment in the production namespace

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     inputDetail:
       type: container_stdout
       containerStdout:
         allContainers: false
         workloads:
         - namespace: production
           name: ingress-gateway
           kind: deployment
     ...
    

    Sample 3: collecting the container standard output in the Pod whose Pod labels contain "k8s-app=nginx" in the production namespace

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     inputDetail:
       type: container_stdout
       containerStdout:
         namespace: production
         allContainers: false
         includeLabels:
           k8s-app: nginx
     ...
    

    Step 3. Create a LogConfig object

    Since the LogConfig.yaml declaration file is defined in Step 2. Define the LogConfig object, you can run the kubectl command to create a LogConfig object.

    kubectl create -f /usr/local/LogConfig.yaml
    

    Step 4. Configure CLS authentication ConfigMap

    To upload logs from a self-built Kubernetes cluster to CLS, you need to create a ConfigMap for storing the API key ID and API key.

    1. Run the wget command to download the ConfigMap.yaml declaration file, using the master node path /usr/local/ as an example.
      wget https://mirrors.tencent.com/install/cls/k8s/ConfigMap.yaml
      
    Note:

    During configuration, set TmpSecretId and TmpSecretKey in ConfigMap.yaml to your API key ID and API key respectively.
    2. Run the kubectl command to create a ConfigMap object.

    kubectl create -f /usr/local/ConfigMap.yaml
    

    Step 5. Deploy Log-Provisioner

    Log-Provisioner discovers and listens for CLS consumer information, log collection rules, and log file paths from LogConfig resources and synchronizes them to CLS. 

    1. Run the wget command to download the Log-Provisioner.yaml declaration file, using the master node path /usr/local/ as an example.
      wget https://mirrors.tencent.com/install/cls/k8s/Log-Provisioner.yaml
      
    Note:

    During configuration, set the env environment variable field CLS_HOST in Log-Provisioner.yaml to the domain name of the region where the target log topic belongs. For the domain names of different regions, see Available Regions. In addition, the env environment variable field CLUSTER_ID must be set to any name that is different from the names of all machine groups under your account. You can view all machine groups under your account on the machine group management page in the CLS console.

    1. Use kubectl to deploy Log-Provisioner in Deployment mode.
      kubectl create -f /usr/local/Log-Provisioner.yaml
      

    Step 6. Deploy Log-Agent and LogListener

    Cluster log collection requires two components:

    • Log-Agent: pulls the information of log sources specified in the cluster LogConfig and calculates the absolute paths of the container logs mapped on hosts.
    • LogListener: collects and parses log files in host log file paths and uploads them to CLS.
    1. Run the wget command to download the declaration files of Log-Agent and LogListener, using the master node path /usr/local/ as an example.
      wget https://mirrors.tencent.com/install/cls/k8s/Log-Agent.yaml
      
    Note:

    • During configuration, set the env environment variable field CLS_HOST in Log-Agent.yaml to the domain name of the region where the target log topic belongs. For the domain names of different regions, see Available Regions. In addition, the env environment variable field CLUSTER_ID must be set to any name that is different from the names of all machine groups under your account. You can view all machine groups under your account on the machine group management page in the CLS console.
    • If the docker root directory of the host is not under /var/lib/docker (root directory of the host), you need to map the docker root directory to the container in the Log-Agent.yaml declaration file. As shown in the following figure, mount /data/docker to the container.
    1. Run the kubectl command to deploy Log-Agent and LogListener in DaemonSet mode.
    kubectl create -f /usr/local/Log—Agent.yaml
    

    Subsequent Operations

    After the deployment for cluster log collection is completed, you can go to CLS console > Search and Analysis to view collected logs.

    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support