tencent cloud

Feedback

Using CRD to Configure Log Collection via YAML

Last updated: 2022-03-23 10:54:16

    Overview

    Besides using the TKE console, you can also configure the log collection settings by using the Custom Resource Definitions (CRD). CRD supports the collection of container standard outputs, container files, and host files. It also supports multiple log collection formats, and supports shipping logs to different consumers such as CLS and CKafka.

    Prerequisites

    You have enabled Log Collection in Feature Management in the TKE console. For more information, see Enabling Log Collection.

    Creating CRD to Ship Logs to CLS

    To create a collection configuration, you only need to define the LogConfig CRD. The log-agent modifies the corresponding CLS log topics based on changes to the LogConfig CRD and set the bound server group. The CRD format is as follows:

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig                              ## Default value
    metadata:
     name: test                                ## CRD resource name, unique in the cluster
    spec:
     clsDetail:
       # Note: You cannot modify the topic after it is specified.
       # If the log topic is created automatically, the names of logset and topic need to be specified at the same time.
       logsetName: test                        ## CLS logset name. Logset for the name will be created automatically if there is not any. If there is the logset, log topic will be created under it.
       topicName: test                         ## CLS log topic name. Log topic for the name will be created automatically if there is not any.
        
       # Select an existing log topic
       topicId: xxxxxx-xx-xx-xx-xxxxxxxx       ## CLS log topic ID. The log topic needs to be created in CLS in advance and should not be occupied by other collection configurations.
       logType: minimalist_log                 ## Log collection format. json_log: json format. delimiter_log: separator-based format. minimalist_log: full text in a single line. multiline_log: full text in multi lines. fullregex_log: full regex format.
       extractRule:                            ## Extraction and filtering rule
         ...
     inputDetail:
       type: container_stdout                  ## Log collection type, including container_stdout (container standard output), container_file (container file), and host_file (host file)
       
       containerStdout:                        ## Container standard output
         namespace: default                    ## The Kubernetes namespace of the container to be collected. If this parameter is not specified, it indicates all namespaces.
         allContainers: false                  ## Whether to collect the standard output of all containers in the specified namespace
         container: xxx                        ## Name of the container of which the logs will be collected. If the name is empty, it indicates the log names of all matching containers will be collected.
         includeLabels:                         ## Only Pods that contain the specified labels will be collected.
           k8s-app: xxx                        ## Only the logs generated by Pods with the configuration of "k8s-app=xxx" in the Pod labels will be collected. This parameter cannot be specified at the same time as workloads and allContainers=true.
         workloads:                            ## Kubernetes workload to which the container Pod to be collected belongs
         - namespace: prod                     ## Workload namespace
           name: sample-app                    ## Workload name
           kind: deployment                    ## Workload type. Supported values include deployment, daemonset, statefulset, job, and cronjob.
           container: xxx                      ## Name of the container to collect. If this parameter is not specified, it indicates all containers in the workload Pod will be collected.
    
       containerFile:                          ## File in the container
         namespace: default                    ## The Kubernetes namespace of the container to be collected. A namespace must be specified.
         container: xxx                        ## The name of container of which the logs will be collected. The * indicates the log names of all matching containers will be collected.
        includeLabels:                         ## Only Pods that contain the specified labels will be collected.
           k8s-app: xxx                        ## Only the logs generated by Pods with the configuration of "k8s-app=xxx" in the Pod labels are collected. This parameter cannot be specified at the same time as workload.
         workload:                             ## Kubernetes workload to which the container Pod to be collected belongs
           name: sample-app                    ## Workload name                  
           kind: deployment                    ## Workload type. Supported values include deployment, daemonset, statefulset, job, and cronjob.
         logPath: /opt/logs                    ## Log folder. Wildcards are not supported.
         filePattern: app_*.log                ## Log file name. It supports the wildcards "*" and "?". "*" matches multiple random characters, and "?" matches a single random character.
        
       hostFile:                               ## Host file
         logPath: /opt/logs                    ## Log folder. Wildcard is supported.
         filePattern: app_*.log                ## Log file name. It supports the wildcards "*" and "?". "*" matches multiple random characters, and "?" matches a single random character.
         customLablels:
           k1: v1
    
    Note

    If the collection type is selected as "Container File Path", the corresponding path cannot be a soft link. Otherwise, the actual path of the soft link will not exist in the collector's container, resulting in log collection failure.

    Configuring the CLS log parsing format

    A log with full text in a single line means a line is a full log. When CLS collects logs, it uses the line break \n to mark the end of a log. For easier structural management, a default key value __CONTENT__ is given to each log, but the log data itself will no longer be structured, nor will the log field be extracted. The time attribute of a log is determined by the collection time. For more information, see Full Text in a Single Line.

    Assume that the raw data of a log is as follows:

    Tue Jan 22 12:08:15 CST 2019 Installed: libjpeg-turbo-static-1.2.90-6.el7.x86_64
    

    A sample of LogConfig configuration is as follows:
    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     clsDetail:
       topicId: xxxxxx-xx-xx-xx-xxxxxxxx
       # Single-line log
       logType: minimalist_log
    

    The data collected to CLS is as follows:
    __CONTENT__:Tue Jan 22 12:08:15 CST 2019 Installed: libjpeg-turbo-static-1.2.90-6.el7.x86_64
    

    Creating CRD to Ship Logs to CKafka

    You can configure CRD to ship the logs of Pods in TKE to external Kafka service or Tencent Cloud CKafka. You need to define the log source and consumer according to the following configurations.
    The specific configuration of CRD is as follows:

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig                          ## Default value
    metadata:
     name: test                                ## CRD resource name, unique in the cluster
    spec:
     kafkaDetail:
       kafkaType:       ## (Required) Kafka service type. "ckafka" indicates Tencent Cloud CKafka and "" indicates external Kafka services.
         type: string
       instanceId:     ## ckafka instance ID.
         type: string
       brokers: xxxxxx       ## (Required) The broker address. Generally, it is domain name:port. If there are more than one address, separate them with ",".
       topic: xxxxxx         ## (Required) Topic name.        
       messageKey:           ## (Optional) You can specify the Pod field as the key to upload to the specified partition.
         valueFrom:
           fieldRef:
             fieldPath: metadata.name   
       timestampKey:            ## The key of timestamp. Default value is @timestamp.
       timestampFormat:       ## The format of timestamp. Default value is double.
     inputDetail:
       type: container_stdout                  ## Log collection type, including container_stdout (container standard output) and container_file (container file).
        containerStdout:                        ## Container standard output
         namespace: default                    ## The Kubernetes namespace of the container to be collected. If this parameter is not specified, it indicates all namespaces.
         allContainers: false                  ## Whether to collect the standard output of all containers in the specified namespace
         container: xxx                        ## Name of the container to be collected. This item can be left empty.
         includeLabels:                         ## Only Pods that contain the specified labels will be collected.
           k8s-app: xxx                        ## Only the logs generated by Pods with the configuration of "k8s-app=xxx" in the Pod labels will be collected. This parameter cannot be specified at the same time as workloads and allContainers=true.
         workloads:                            ## Kubernetes workload to which the container Pod to be collected belongs
         - namespace: prod                     ## Workload namespace
           name: sample-app                    ## Workload name
           kind: deployment                    ## Workload type. Supported values include deployment, daemonset, statefulset, job, and cronjob.
           container: xxx                      ## Name of the container to be collected. If this item is left empty, it indicates all containers in the workload Pod will be collected.
        containerFile:                          ## File in the container
         namespace: default                    ## The Kubernetes namespace of the container to be collected. A namespace must be specified.
         container: xxx                        ## Name of the container to be collected. You can enter a * for this item.
         includeLabels:                         ## Only Pods that contain the specified labels will be collected.
           k8s-app: xxx                        ## Only the logs generated by Pods with the configuration of "k8s-app=xxx" in the Pod labels are collected. This parameter cannot be specified at the same time as workload.
         workload:                             ## Kubernetes workload to which the container Pod to be collected belongs
           name: sample-app                    ## Workload name                  
           kind: deployment                    ## Workload type. Supported values include deployment, daemonset, statefulset, job, and cronjob.
         logPath: /opt/logs                    ## Log folder. Wildcards are not supported.
         filePattern: app_*.log                ## Log file name. It supports the wildcards "*" and "?". "*" matches multiple random characters, and "?" matches a single random character.
        
         ...
    

    Log Collection Types

    Container standard output

    Sample 1: collecting the standard output of all containers in the default namespace

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     inputDetail:
       type: container_stdout
       containerStdout:
         namespace: default
         allContainers: true
    ...
    

    Sample 2: collecting the container standard output in the Pod that belongs to ingress-gateway deployment in the production namespace

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     inputDetail:
       type: container_stdout
       containerStdout:
         allContainers: false
         workloads:
         - namespace: production
           name: ingress-gateway
           kind: deployment
     ...
    

    Sample 3: collecting the container standard output in the Pod whose Pod labels contain “k8s-app=nginx” under the production namespace

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     inputDetail:
       type: container_stdout
       containerStdout:
         namespace: production
         allContainers: false
         includeLabels:
           k8s-app: nginx
     ...
    

    Container file

    Sample 1: collecting the access.log file in the /data/nginx/log/ path in the nginx container in the Pod that belongs to ingress-gateway deployment under the production namespace

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     topicId: xxxxxx-xx-xx-xx-xxxxxxxx
     inputDetail:
       type: container_file
       containerFile:
         namespace: production
         workload:
           name: ingress-gateway
           type: deployment
         container: nginx
         logPath: /data/nginx/log
         filePattern: access.log
     ...
    

    Sample 2: collecting the access.log file in the /data/nginx/log/ path in the nginx container in the Pod whose pod labels contain “k8s-app=ingress-gateway” under the production namespace

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     inputDetail:
       type: container_file
       containerFile:
         namespace: production
         includeLabels:
           k8s-app: ingress-gateway
         container: nginx
         logPath: /data/nginx/log
         filePattern: access.log
     ...
    

    Host file

    Sample: collect all .log files in the host path /data/

    apiVersion: cls.cloud.tencent.com/v1
    kind: LogConfig
    spec:
     inputDetail:
       type: host_file
       hostFile:
         logPath: /data
         filePattern: *.log
     ...
    

    Metadata

    For container standard output (container_stdout) and container files (container_file), in addition to the raw log content, the container metadata (for example, the ID of the container that generated the logs) also needs to be carried and reported to CLS. In this way, when viewing logs, users can trace the log source or search based on the container identifier or characteristics (such as container name and labels).
    The following table lists the metadata:

    Field Name Description
    cluster_id The ID of the cluster to which logs belong
    container_name The name of the container to which logs belong
    container_id ID of the container to which logs belong
    image_name The image name IP of the container to which logs belong
    namespace The namespace of the Pod to which the logs belong
    pod_uid The UID of the Pod to which the logs belong
    pod_name The name of the Pod to which the logs belong
    pod_ip The IP of the Pod to which the logs belong
    pod_lable_{label name} The labels of the Pod to which the logs belong (for example, if a Pod has two labels: app=nginx and env=prod, the reported log will have two metadata entries attached: pod_label_app:nginx and pod_label_env:prod).
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support