Storing Access Logs in CLS
Last updated: 2020-05-22 14:42:38PDF
CLB supports configuring layer-7 (HTTP/HTTPS) access logs that can help you better understand client requests, troubleshoot issues, and analyze user behaviors. Currently, access logs can be stored in CLS, reported at a minute granularity, and searched online by multiple rules.
Access logs of CLB are mainly used to quickly locate and troubleshoot issues. The access logging feature includes log reporting, storage, and search:
- Log reporting provides best-effort service, that is, it prioritizes service forwarding over log reporting.
- Log storage and search provide SLA based on the storage service currently in use.
- As the feature of storing access logs in COS will be officially disused after 00:00:00, June 30, 2020, you are recommended to use CLS for CLB access log storage.
- Currently, access logs can be stored in CLS only for layer-7 protocols (HTTP/HTTPS) but not layer-4 protocols (TCP/UDP/TCP SSL).
- The access log service for CLB is free of charge, and you only need to pay for CLS usage.
- Currently, access logs can be stored in CLS in the Guangzhou, Shanghai, Chengdu, Hong Kong (China), and Toronto regions through the console or APIs. This feature is in beta test in the Nanjing, Beijing, Chongqing, Singapore, Mumbai, and Silicon Valley regions. To try it out, please submit a ticket for application.
- Currently, CLS is in beta test. To try it out, please submit a ticket for application.
Enabling Access Logging
- Log in to the CLB Console.
- Click the ID of the CLB instance to be configured to enter the "Basic Information" page.
- In the "Log Access" module, edit "Cloud Log Service".
- In the pop-up box, enable access logging and select the destination logset and log topic for access log storage. If you haven't created a logset or log topic yet, please create relevant resources and then select them as the storage location.
- Click Submit and access logs will be collected into the corresponding topic.
- Then, click the logset or log topic to redirect to the log search page in CLS.
- (Optional) If you want to disable access logging, you can edit "Cloud Log Service" again to disable it and submit in the pop-up window.
Searching for Access Logs
Step 1. Configure log topic indexes
The log topics must be configured with indexes; otherwise, no logs can be searched for.
- Log in to the CLS Console.
- On the left sidebar, click Logset to enter the "Logset Management" page.
- Click a logset ID to enter the logset details page.
- On the logset details page, click a log topic ID to enter the log topic details page.
- On the log topic details page, select the Index Configuration tab. You can select some variables from the log variables and configure the index fields as needed. For more information on how to configure, please see Enabling Index.
- The result of index configuration is as shown below:
Step 2. Search for access logs
- Log in to the CLS Console.
- On the left sidebar, click Search and Analysis to enter the "Search Analysis" page.
- On the "Search Analysis" page, select a logset, log topic, and time range, and click Search Analysis to search for the access logs reported by CLB to CLS. For more information on the search syntax, please see Syntax and Rules.
Log Variable Description
|time_local||Access time and time zone, such as "01/Jul/2019:11:11:00 +0800" where "+0800" represents UTC+8, i.e., Beijing time.|
|protocol_type||Protocol type (HTTP/HTTPS/SPDY/HTTP2/WS/WSS).|
|server_addr||Destination IP of request.|
|server_port||Destination port of request.|
|server_name||Rule's `server_name`, i.e., server name.|
|status||Status code returned to client.|
|upstream_status||Status code returned by RS to CLB.|
|request_length||Number of bytes of request received from client.|
|bytes_sent||Number of bytes sent to client.|
|http_host||Request domain name.|
|http_user_agent||`user_agent` field of the HTTP header.|
|http_referer||HTTP request source.|
|request_time||Request processing time. The timing begins when the first byte is received from the client and stops when the last byte is sent to the client, i.e., the total time the whole process takes, where the client request reaches a CLB instance, the CLB instance forwards the request to an RS, the RS responds and sends data to the CLB instance, and finally the CLB instance forwards the data to the client.|
|upstream_response_time||The time that an entire backend request process takes. The timing begins when a CLB instance connects with an RS and stops when the RS receives the request and responds.|
|upstream_connect_time||The time it takes to establish a TCP connection with an RS. The timing begins when a CLB instance connects with an RS and stops when it sends the HTTP request.|
|upstream_header_time||The time it takes to receive an HTTP header from the RS. The timing begins when a CLB instance connects with an RS and stops when the HTTP response header is received from the RS.|
|tcpinfo_rtt||TCP connection RTT.|
|connection_requests||Number of requests on connection.|
|ssl_handshake_time||The time that an SSL handshake takes.|
|ssl_cipher||SSL cipher suite.|
|ssl_protocol||SSL protocol version.|
|vip_vpcid||VPC ID of CLB instance VIP.|