Storing Access Logs in CLS

Last updated: 2020-05-22 14:42:38

PDF

CLB supports configuring layer-7 (HTTP/HTTPS) access logs that can help you better understand client requests, troubleshoot issues, and analyze user behaviors. Currently, access logs can be stored in CLS, reported at a minute granularity, and searched online by multiple rules.

Access logs of CLB are mainly used to quickly locate and troubleshoot issues. The access logging feature includes log reporting, storage, and search:

  • Log reporting provides best-effort service, that is, it prioritizes service forwarding over log reporting.
  • Log storage and search provide SLA based on the storage service currently in use.
  • As the feature of storing access logs in COS will be officially disused after 00:00:00, June 30, 2020, you are recommended to use CLS for CLB access log storage.
  • Currently, access logs can be stored in CLS only for layer-7 protocols (HTTP/HTTPS) but not layer-4 protocols (TCP/UDP/TCP SSL).
  • The access log service for CLB is free of charge, and you only need to pay for CLS usage.
  • Currently, access logs can be stored in CLS in the Guangzhou, Shanghai, Chengdu, Hong Kong (China), and Toronto regions through the console or APIs. This feature is in beta test in the Nanjing, Beijing, Chongqing, Singapore, Mumbai, and Silicon Valley regions. To try it out, please submit a ticket for application.
  • Currently, CLS is in beta test. To try it out, please submit a ticket for application.

Enabling Access Logging

  1. Log in to the CLB Console.
  2. Click the ID of the CLB instance to be configured to enter the "Basic Information" page.
  3. In the "Log Access" module, edit "Cloud Log Service".
  4. In the pop-up box, enable access logging and select the destination logset and log topic for access log storage. If you haven't created a logset or log topic yet, please create relevant resources and then select them as the storage location.
  5. Click Submit and access logs will be collected into the corresponding topic.
  6. Then, click the logset or log topic to redirect to the log search page in CLS.
  7. (Optional) If you want to disable access logging, you can edit "Cloud Log Service" again to disable it and submit in the pop-up window.

Searching for Access Logs

Step 1. Configure log topic indexes

The log topics must be configured with indexes; otherwise, no logs can be searched for.

  1. Log in to the CLS Console.
  2. On the left sidebar, click Logset to enter the "Logset Management" page.
  3. Click a logset ID to enter the logset details page.
  4. On the logset details page, click a log topic ID to enter the log topic details page.
  5. On the log topic details page, select the Index Configuration tab. You can select some variables from the log variables and configure the index fields as needed. For more information on how to configure, please see Enabling Index.
  6. The result of index configuration is as shown below:

Step 2. Search for access logs

  1. Log in to the CLS Console.
  2. On the left sidebar, click Search and Analysis to enter the "Search Analysis" page.
  3. On the "Search Analysis" page, select a logset, log topic, and time range, and click Search Analysis to search for the access logs reported by CLB to CLS. For more information on the search syntax, please see Syntax and Rules.

Log Variable Description

VariableDescription
stgw_request_idRequest ID.
time_local Access time and time zone, such as "01/Jul/2019:11:11:00 +0800" where "+0800" represents UTC+8, i.e., Beijing time.
protocol_typeProtocol type (HTTP/HTTPS/SPDY/HTTP2/WS/WSS).
server_addrDestination IP of request.
server_portDestination port of request.
server_nameRule's `server_name`, i.e., server name.
remote_addrClient IP.
remote_portClient port.
statusStatus code returned to client.
upstream_addrRS address.
upstream_statusStatus code returned by RS to CLB.
proxy_hostStream ID.
requestRequest line.
request_lengthNumber of bytes of request received from client.
bytes_sentNumber of bytes sent to client.
http_hostRequest domain name.
http_user_agent`user_agent` field of the HTTP header.
http_refererHTTP request source.
request_timeRequest processing time. The timing begins when the first byte is received from the client and stops when the last byte is sent to the client, i.e., the total time the whole process takes, where the client request reaches a CLB instance, the CLB instance forwards the request to an RS, the RS responds and sends data to the CLB instance, and finally the CLB instance forwards the data to the client.
upstream_response_timeThe time that an entire backend request process takes. The timing begins when a CLB instance connects with an RS and stops when the RS receives the request and responds.
upstream_connect_timeThe time it takes to establish a TCP connection with an RS. The timing begins when a CLB instance connects with an RS and stops when it sends the HTTP request.
upstream_header_timeThe time it takes to receive an HTTP header from the RS. The timing begins when a CLB instance connects with an RS and stops when the HTTP response header is received from the RS.
tcpinfo_rttTCP connection RTT.
connectionConnection ID.
connection_requestsNumber of requests on connection.
ssl_handshake_timeThe time that an SSL handshake takes.
ssl_cipherSSL cipher suite.
ssl_protocolSSL protocol version.
vip_vpcidVPC ID of CLB instance VIP.