CLS Connection to Grafana

Last updated: 2021-04-14 10:32:19

    Overview

    CLS can be connected to Grafana to export the raw log data and SQL aggregate analysis results for display in Grafana. To display the results in Grafana, you only need to install the Grafana plugins for CLS and enter search and analysis statements in Grafana.

    You can log in to Grafana by using the following username and password for trial.

    • Username: Viewer
    • Password: clsdemo

    This document describes how to install and configure Grafana on CentOS.

    Directions

    Installing Grafana

    For more information on how to install Grafana, please see Install Grafana.
    The following uses installing Grafana 7.3.6 on CentOS as an example:

    sudo yum install initscripts urw-fonts wget
    wget https://dl.grafana.com/oss/release/grafana-7.3.6-1.x86_64.rpm
    sudo yum install grafana-7.3.6-1.x86_64.rpm
    sudo systemctl daemon-reload
    sudo systemctl start grafana-server
    sudo systemctl status grafana-server 
    sudo systemctl enable grafana-server  
    

    To install more visual panels (such as pie and graph panels), please install Grafana panel plugins by running the corresponding commands.
    For example, if you want to install the pie panel, you can run the following command:

    grafana-cli plugins install grafana-piechart-panel
    service grafana-server restart
    

    For information on more plugins, please see Grafana Plugins.

    Installing and configuring Grafana plugins for CLS

    1. Install Grafana plugins for CLS in the /var/lib/grafana/plugins/ plugin directory.
      cd /var/lib/grafana/plugins/
      wget https://github.com/TencentCloud/cls-grafana-datasource/releases/latest/download/cls-grafana-datasource.zip
      unzip cls-grafana-datasource.zip
      

      Note:

      • If your CVM instance is not on CentOS, please confirm the location of the Grafana plugin directory first and go to the directory for installation.
      • You need to install Grafana 7.0 or above. If your Grafana version is lower than 7.0, configuration backup and upgrade are required. For details, see Upgrade Grafana.
    2. Open the grafana.ini configuration file on the server where Grafana has been deployed.
    • The file path on macOS is /usr/local/etc/grafana/grafana.ini.
    • The file path on Linux is /etc/grafana/grafana.ini.
    1. Set the allow_loading_unsigned_plugins parameter in plugins.
      allow_loading_unsigned_plugins = tencent-cls-grafana-datasource
      
    2. Run the following command to restart the Grafana service:
      service grafana-server restart
      

      Configuring log data source

    3. Log in to Grafana by accessing the following URL from your browser.

      Note:

      The default port of Grafana is 3000.

      http://Grafana IP address: 3000
      
    4. On the left sidebar, select the Settings icon to go to the Data Sources page.
    5. On the Data Sources page, click Add data source.
    6. Select Tencent Cloud Log Service Datasource and configure the data source as instructed below.
      image-20201229200229285
      Configuration ItemDescription
      Security Credentials**SecretId** and **SecretKey**: API request key, which is used for authentication. You can go to the API Key Management page to get a key.
      Log Service Info
      • **Region**: abbreviation of the CLS region. For example, enter `ap-beijing` for the Beijing region.
        For the complete list of regions, please see Available Regions.
      • **TopicId**: log topic ID.

    Configuring dashboard

    1. On the left sidebar, click Creat Dashboards.
    2. On the dashboard page, click Add new panel.
    3. Select the log data source you just created as the data source as shown below:
      image-20201229200254913
    4. Enter the query statement, select the format according to the panel type to be displayed, and the system will automatically convert the data for display in Grafana.
      FormatDescriptionConfiguration Item
      Log panelLog panel is used to shown log search results. Query syntax supports searching by keyword and fuzzy match. For more information, see [Syntax and Rules](https://intl.cloud.tencent.com/document/product/614/30439). Eg. status:400**limit**: specifies the number of log search results to be returned.
      Table panelTable panel automatically shows the results of whatever columns and rows your query returns.None
      Graph,Pie,Gauge panelIn this pattern, there is a format transformation where data will be adapted to graph,pie,gauge panel.
      • **Metrics**: metrics to be collected.
      • **Bucket**: (optional) name of the aggregate column.
      • **Time**: (optional) if the result returned by a query is continuous time data, you need to specify the **Time** field; otherwise, leave it empty.

    Samples

    Graph

    A graph shows the PV and UV curves as shown below:

    You can configure it according to the following information:

    • The query statement is entered as shown below:
    * | select histogram( cast(__TIMESTAMP__ as timestamp),interval 1 minute) as time, count(*) as pv,count( distinct remote_addr) as uv group by time order by time limit 1000
    
    • Format: select Graph,Pie,Gauge panel.
    • Metrics: pv, uv.
    • Bucket: leave it empty if there are no aggregate columns.
    • Time: time.

    Pie

    A pie shows the distribution of request status codes as shown below:

    You can configure it according to the following information:

    • The query statement is entered as shown below:
    * | select count(*) as count, status group by status
    
    • Format: select Graph,Pie,Gauge panel.
    • Metrics: count.
    • Bucket: status.
    • Time: leave it empty if it is not continuous time data.

    Bar gauge

    A bar gauge shows the top 10 pages in terms of access latency as shown below:

    You can configure it according to the following information:

    • The query statement is entered as shown below:
    * | select http_referer,avg(request_time) as lagency group by http_referer order by lagency desc limit 10
    
    • Format: select Graph,Pie,Gauge panel.
    • Metrics: latency.
    • Bucket: http_referer.
    • Time: leave it empty if it is not continuous time data.

    Table

    A table shows the top 10 users in terms of access requests as shown below:

    You can configure it according to the following information:

    • The query statement is entered as shown below:
    * | select remote_addr,count(*) as count group by remote_addr order by count desc limit 10
    
    • Format: Table.