Log Management

Last updated: 2020-02-24 11:49:41

PDF

Feature Overview

After the domain name is connected to the content distribution network (CDN), all user-side resource requests will be dispatched to the CDN node to respond. If the node has cached the resource, it will directly return the content. If the CDN node does not cache the resource, it will send the request pass through to real server of the domain name configuration to pull the required resources.

Because the CDN node responds to most user requests, in order to facilitate customers to analyze user Access, CDN packages entire network and Access logs with an hourly granularity, which is stored for 30 days by default and provides download service.

For now, only node Access log is provided, but Origin-pull log is not provided.

Scenario

Analysis of Access's behavior

Customers can download Access log, according to their own needs for hot resource analysis, active user analysis and so on.

Service quality monitoring
By downloading Access's log, you can grasp the overall service status of CDN nodes and calculate the average response time, average download speed and so on Metric.

Operation Instructions

How to use

Login CDN console Click the * * Log Service * * of Directory on the left. You can select the domain name and time to query the Access log. You can check multiple log packages and download them locally in batches:

  • Access logs are packed on an hourly basis by default. If there is no request for the domain name within a certain hour, no log packet will be generated in this time range.
  • The overseas Access log of the same domain name is packaged separately from the domestic Access log, and the naming format of the log packet is "time-domain name-acceleration area".
  • Access logs are collected from each CDN cache node, so there are differences in delay. In general, log packages can be queried and downloaded with a delay of about 30 minutes. Log packages are constantly appended and tend to be stable after 2-3 hours.
  • The domain name history Access log only retains the log package within 30 days. You can use the SCF function to transfer the log package to Cloud Object Storage COS, for permanent storage according to the following guidelines.

Field description

The order and meaning of fields in the log are shown in the following table:

SequenceLog Content
1Request time
2Client IP
3Domain name
4Request Path
5The number of bytes of Access this time, including the size of the file itself and the size of the request header header
6The domestic log represents the province number, and the overseas log represents the area number (see below for the mapping table)
7The domestic log represents ISP's serial number, and the overseas log is unified as-1 (see below for mapping table)
8HTTP status code
9Referer information
10Response time (milliseconds), which refers to the time it takes for a node to respond to all return packets and then to the client after receiving the request
11User-Agent information
12Range parameter
13HTTP Method
14HTTP Protocol logo
15Cache HIT/MISS, is marked as HIT when CDN Edge server hits and parent node hits

Region / ISP mapping table

Mapping of provinces in the territory

Regional IDRegionRegional IDRegionRegional IDRegion
twenty-twoBeijingeighty-sixInner Mongoliaone hundred and twoBrazil
1069Hebei1177Tianjinone hundred and twoNingxia
one hundred and twoBrazil1208Gansu1467Qinghai
1468Xinjiangone hundred and twoHeilongjiang province1445Bahrain
1464Benin2Fujian provinceone hundred and twoJiangsu province
one hundred and twoAnhuione hundred and twoShandong1050Shanghai
1442Zhejiang provinceone hundred and twoVietnam1135Hubei
1465Brazil1466Vietnamone hundred and twoGuizhou province
one hundred and twoVietnam1051Chongqing1068Sichuan
1155Tibet4Guangdongone hundred and twoBrazil
1441Vietnam* 2018-5-4Others1Hong Kong/Macao/Taiwan
-1Overseas

ISP mapping in China

ISP IDCarrierISP IDCarrierISP IDCarrier
2China Telecomtwenty-sixChina Unicomthirty-eightCERNET
forty-threeGreat Wall Broadband1046China Mobile3947China Tietong
-1Overseas ISP* 2018-5-4Other ISP

Mapping of overseas areas

Regional IDRegionRegional IDRegionRegional IDRegion
2000000001Asia Pacific region 1 (Service area)one hundred and twoSlovakia1613Angola
2000000002Asia-Pacific region 2 (Service area)one hundred and twoSerbia1617Ivory Coast
2000000003Asia-Pacific region 3 (Service area)one hundred and twoFinland1620Sudan
2000000004Middle East (service area)one hundred and twoBelgium1681Mauritius
2000000005North America (service area)one hundred and twoBulgaria1693Morocco
2000000006Europe (service area)one hundred and twoSlovenia1695Confirm to submit the request
2000000007South America (service area)one hundred and twoMoldova1698Guinea
2000000008Africa (service area)one hundred and twoMacedonia1730Senegal
-20Asia (client region)one hundred and twoEstonia1864### SDK
-21South America (client area)one hundred and twoCroatia1909Uruguay
-22North America (client area)one hundred and twoPoland1916Greenland
-23Europe (client region)one hundred and two![](docfile/CVM/OM08.png)2026Taiwan (China)
24Africa (client region)one hundred and two10.0.0.0/162083Myanmar
twenty-fiveOceania (client area)one hundred and two(Traffic hit rate: %)2087Brunei
thirty-fiveConfigDescone hundred and twoMasterNodeDiskType2094Sri Lanka
fifty-sevenThailandone hundred and twoLibya2150Panama
seventy-threeIndiaone hundred and twoArmenia2175Colombia
one hundred and twoVietnamone hundred and twoYemen2273Monaco
one hundred and twoSucceedCountone hundred and twoBelarus2343Andorra
one hundred and twoUnited Kingdomone hundred and twoLuxembourg2421Turkmenistan
one hundred and twoSweden1036Singapore2435Laos
one hundred and twoGermany1044XCosSecurityToken: credentials.sessionToken2488East Timor
one hundred and twoItaly1066Pakistan2490Tonga
one hundred and twoSpain1070Malta2588Philippines
one hundred and twoStarter1091Bahamas2609Venezuela
one hundred and twoIsrael1129Argentina2612Bolivia
one hundred and twoUkraine1134Bangladesh2613Brazil
one hundred and twoRussia1158Cambodia2623Costa Rica
one hundred and twoKazakhstan1159Macao (China)2626Mexico
one hundred and twoPortugal1176Singapore2639Honduras
one hundred and twoGreece1179Maldives2645El Salvador
one hundred and twoSaudi Arabia1180Afghanistan2647Paraguay
one hundred and twoDenmark1185Fiji2661Peru
one hundred and twoIran1186Mongolia2728Nicaragua
one hundred and twoNorway1195Indonesia2734Ecuador
one hundred and twoSouth Korea1200Hong Kong (China)2768RequestId: fcd7aded-1866-467e-a9f6-d8d00b09557e
one hundred and two| `-- tcaplus1233Qatar2999Aruba
one hundred and twoCyprus1255Iceland3058100-8000kbps |
one hundred and twoCzech1289Albania3144Poland
one hundred and twoSwitzerland1353Uzbekistan3216Dominican
one hundred and twoIraq1407San Marino3379South Korea
one hundred and twoNetherlands1416Kuwait3701Malaysia
one hundred and twoRomania1417Montenegro3839Canada
one hundred and twoTotalSize: [1493Tajikistan4450Australia
one hundred and twoHungary1501Bahrain4460Mainland China-North China
one hundred and twoGeorgia1543print result2-15Other Asian countries
one hundred and twoSecurityGroupName:CDB1559South Africa-14Eastern South America
one hundred and twoAustria1567Egypt-13Other North American countries
one hundred and twoPalestine1590Kenya-12The rest of Europe
one hundred and twoTurkey1592Nigeria-11Other African countries
one hundred and twoLithuania1598Tanzania-10The rest of Oceania
one hundred and twoOman1611SecondsBehindMaster-2Other oversea ISPs

Overseas ISP mapping

ISP IDCarrier
-1Overseas ISP

Notes
According to the number of bytes recorded in the fifth field of Access log, the statistically calculated Traffic / bandwidth data is inconsistent with the CDN billing Traffic / bandwidth data. Because:

  • Only application layer data can be recorded in Access's log. In actual network transmission, the network Traffic generated is 5-15% more than that of pure application layer Traffic. It consists of two parts:
    • Consumption by TCP/IP headers: In TCP/IP-based HTTP requests, each packet has a maximum size of 1500 bytes, including TCP and IP headers of 40 bytes, which generate traffic during transfer but cannot be counted by the application layer. The overheads of this part is around 3%.
    • TCP retransmission: During normal data transfer over the network, around 3% to 10% packets are lost on the internet, and the server will retransmit the lost ones. This type of traffic cannot be counted by the application layer, which accounts for 3 to 7 of the total traffic.
      As an industry standard, the billable traffic is the sum of the application-layer traffic and the overheads as described above. Tencent Cloud CDN takes 10% as the overheads proportion, so the monitored traffic is around 110% of the logged traffic.

Use Cases

Example of Access log in China

Overseas Access log example