Real-time Log Analysis enables quick retrieval, analysis, and storage of log data through the real-time collection and delivery of access logs from Cloud Streaming Services to the Cloud Log Service (CLS) product. By mining log data, it supports data-driven operations and management, allowing for the rapid and accurate development of operational strategies.
Note
Real-time Log Analysis now fully supports shipping logs to Cloud Log Service. You are welcome to experience this. This documentation will guide you on how to use the latest real-time log feature. Notes
Log data is collected in real-time, with log search and reporting data stabilizing after three minutes.
Currently, reporting analysis is only available for playback logs. If you have other log management needs, visit Cloud Log Service. After enabling the log delivery feature, ensure that Cloud Log Service is in normal operation, as the suspension of Cloud Log Service will prevent the delivery of logs.
The bandwidth or traffic data recorded in logs is the application layer (HTTP protocol) return data. Due to mechanisms like TCP header consumption and failed retransmissions, it is smaller than the bandwidth or traffic consumption calculated at the TCP layer.
Operation Instruction
Creating a log topic
1. Go to the Cloud Streaming Services console, and select Monitoring > Cloud Log Service > Real-time log analysis, to enter real-time log analysis. 2. For first use, authorization of the Cloud Streaming Services service role is required. Once the role is authorized, you need to agree to the service agreement and click Start. The system will automatically activate the Cloud Log Service (CLS) product and open the real-time log analysis management page.
3. Choose a region and create a log set, click Here on the page to create a new one.
Note:
The region includes Guangzhou and Singapore. Log topics created under the log set in the Guangzhou region can only deliver logs within the Chinese mainland. In contrast, log topics created under the log set in the Singapore region can only deliver logs internationally / to Hong Kong (China), Macao (China), and Taiwan (China).
4. Click Confirm to create a new log set.
5. After successfully creating a log set, click Creating Log Topic to enter the log topic creation page. The newly created log topic will by default be in the process of delivering logs to the Cloud Log Service (CLS) product.
Modifies a log topic
1. Enter the log topic list in real-time log analysis and click Manage in the operation column of the log topic you need to modify.
2. You can enter the log topic editing page to modify the log topic information.
Analyzing a log report
Only log topics of the log type provide report analysis. There are four types of data on the page: Basic Data Analysis, Resource Distribution Analysis, Exception Diagnosis Analysis, and User Analysis.
1. Enter the log topic list in Real-time Log Analysis and click Report on the right side of the log topic you want to view.
2. Enter the log report page to view report data; you can separately view Basic Data, Resource Distribution, Exception Diagnosis, and User Analysis.
Log Search
Log search supports various types of search analysis methods and chart analysis forms. For detailed information, see Log Search and Analysis. Perform log search by log topic as a cell. Select the log topic you need to search for and click Search to enter the log search page.
Stop shipping log topic to CLS
1. Enter the log topic list in Real-time Log Analysis, and click Stop on the right of the log Topic you wish to stop shipping.
2. In the pop-up window, confirm whether to stop shipping and click Confirm to proceed. The status of the corresponding log topic will change to stop shipping, and logs will no longer be shipped to the CLS product.
Deleting a log topic
Note:
Once a log topic is deleted, it cannot be recovered. Please proceed with caution.
If you delete a log topic, you will stop pushing log data of the related domain, and the corresponding log topic in the Cloud Log Service product will also be deleted and all shipped logs will be cleared. Meanwhile, you will no longer be able to use the report associated with that topic.
1. Enter the log topic list in Real-time Log Analysis and click Delete on the right side of the log topic you wish to remove.
2. In the pop-up window, confirm whether to delete the log topic and click OK to proceed.
Log Field
Streaming Log
|
1 | time | Request time |
2 | client_ip | Client IP |
3 | host | Accessed domain name |
4 | url | URL |
5 | size | Streaming bytes size |
6 | country_id | country_id |
7 | prov | Province |
8 | isp | ISP |
9 | streamname | Stream ID |
10 | node_ip | Node IP |
11 | server_region | Server region |
12 | server_country | Server country |
Playback Log
|
1 | type | Playback type: lvb represents Live Video Broadcasting and leb represents Live Event Broadcasting |
2 | time | Request time |
3 | client_ip | Client IP |
4 | host | Accessed domain name |
5 | url | URL |
6 | size | Number of bytes of this access request |
7 | country_id | country_id |
8 | prov | Province |
9 | isp | ISP |
10 | http_code | HTTP status code |
11 | referer | Referer information |
12 | process_time | Processing duration (in milliseconds) |
13 | ua | User-Agent information |
14 | range | Range parameter |
15 | method | HTTP Method |
16 | streamname | Stream ID |
17 | hit | Cache hit/miss |
18 | node_ip | Node IP |
19 | server_region | Server region |
20 | server_country | Server country |
21 | connect_fd | connect_fd (connection port number) |
22 | lost_rate | The packet loss rate, only valid for type=leb |
23 | rtt | rtt, only valid for type=leb |
Definitions
Logset
A log set (Logset) classifies log topics and metric topics; a log set can contain multiple log topics and metric topics. A log set itself does not store any log data, it only facilitates user management of topics. Cloud Streaming Services log sets have the following basic attribute information:
Note
Guangzhou and Singapore regions are currently supported.
Logset Name: The name of a logset.
Retention period: The default retention period for data in the log set is 30 days.
Creation time: Logset creation time.
Log Topic
A log topic (Log Topic) is a basic unit for log data collection, storage, retrieval, and analysis on the Cloud Log Service (CLS) platform. The vast amounts of logs collected are managed by log topic, including the configuration of the collection rules and storage time, log search and analysis, and log download/consumption/delivery.
Log topic features include:
Collect logs to log topics.
Store and manage logs based on log topics.
Search and analyze logs by log topics.
Ship logs to other platforms based on log topics.
Download and consume logs from log topics.
Note
The information above is excerpted from the Cloud Log Service (CLS) product documentation. For more details, see Log Topic and Logset and refer to the Cloud Log Service (CLS) documentation for accurate information.
Was this page helpful?