Download Logs

Last updated: 2021-08-11 11:27:28

    Feature Overview

    After a domain name is connected to CDN, all requests will be scheduled to a CDN node. If the requested resource is cached on the node, the resource will be returned directly; otherwise, the request will be passed to the origin server to pull the requested resource.

    As CDN nodes respond to most user requests, to help you analyze user access, CDN packages access logs of the entire network at an hourly granularity and retains them for 30 days by default. These logs can also be downloaded.

    Note:

    • Currently origin-pull logs are not provided. Only node access logs are provided.
    • ECDN domain name offline logs cannot be queried by regions for now. For more information, see Log Management.

    Overview

    Access behavior analysis

    You can download access logs and analyze popular resources and active users.

    Service quality monitoring

    By downloading access logs, you can stay on top of the service status of all CDN nodes and calculate the average response time, average download speed, and other metrics.

    Directions

    How to use

    Log in to the CDN Console, click Log Service on the left sidebar, and select a domain name and time range to query access logs. You can select multiple log packages and download them in batches:

    Note:

    • The access logs are packaged by hour by default. If there is no request to the domain name for the hour, no log package will be generated for this hour.
    • For the same domain name, logs of accesses from within and outside the Chinese mainland are packaged separately. Log packages are named in the format of "[time]-[domain name]-[acceleration region]".
    • The access logs are collected from each CDN cache node, so the delay may vary. Generally, log packages can be queried and downloaded after about 30 minutes. Log packages will be added continuously and will stabilize after 2–3 hours.
    • The access log packages of a domain name are retained for 30 days. You can use an SCF function to transfer the log packages to COS as instructed in Regularly Storing CDN Logs for permanent storage.

    Fields

    The fields (from left to right) in the logs are listed as below:

    No. Fields
    1 Request time
    2 Client IP
    3 Domain name
    4 Request path
    5 Number of bytes accessed this time, including the size of the file itself and the size of the request header.
    6 Province numbers for Chinese mainland logs; region numbers for logs outside the Chinese mainland (see the mapping table below).
    7 ISP numbers for Chinese mainland logs; `-1` will be used for logs outside the Chinese mainland (see the mapping table below).
    8 HTTP status code
    9 Referer information
    10 Response time (in milliseconds), which refers to the time it takes for a node to return all packets to the client after receiving a request.
    11 User-Agent information
    12 Range parameter
    13 HTTP method
    14 HTTP protocol identifier
    15 Cache hit/miss. A hit in a CDN edge server or parent node will be marked as hit.
    16 Port that connects the client and CDN nodes. Its value will be - if there is no port.

    Region/ISP mappings

    Chinese mainland provinces

    Region ID Region Region ID Region Region ID Region
    22 Beijing 86 Inner Mongolia 146 Shanxi
    1069 Hebei 1177 Tianjin 119 Ningxia
    152 Shaanxi 1208 Gansu 1467 Qinghai
    1468 Xinjiang 145 Heilongjiang 1445 Jilin
    1464 Liaoning 2 Fujian 120 Jiangsu
    121 Anhui 122 Shandong 1050 Shanghai
    1442 Zhejiang 182 Henan 1135 Hubei
    1465 Jiangxi 1466 Hunan 118 Guizhou
    153 Yunnan 1051 Chongqing 1068 Sichuan
    1155 Tibet 4 Guangdong 173 Guangxi
    1441 Hainan 0 Other 1 Hong Kong (China), Macao (China), and Taiwan (China)
    -1 Outside the Chinese mainland

    Chinese mainland ISPs

    ISP ID ISP ISP ID ISP ISP ID ISP
    2 China Telecom 26 China Unicom 38 CERNET
    43 Great Wall Broadband Network 1046 China Mobile 3947 China Mobile Tietong
    -1 ISPs outside the Chinese mainland 0 Other ISPs

    Regions outside the Chinese mainland

    Region ID Region Region ID Region Region ID Region
    2000000001 Asia Pacific Zone 1 (service area) 765 Slovakia 1613 Angola
    2000000002 Asia Pacific Zone 2 (service area) 766 Serbia 1617 Ivory Coast
    2000000003 Asia Pacific Zone 3 (service area) 770 Finland 1620 Sudan
    2000000004 Middle East (service area) 773 Belgium 1681 Mauritius
    2000000005 North America (service area) 809 Bulgaria 1693 Morocco
    2000000006 Europe (service area) 811 Slovenia 1695 Algeria
    2000000007 South America (service area) 812 Moldova 1698 Guinea
    2000000008 Africa (service area) 813 Macedonia 1730 Senegal
    -20 Asia (client area) 824 Estonia 1864 Tunisia
    -21 South America (client area) 835 Croatia 1909 Uruguay
    -22 North America (client area) 837 Poland 1916 Greenland
    -23 Europe (client area) 852 Latvia 2026 Taiwan (China)
    -24 Africa (client area) 857 Jordan 2083 Myanmar
    -25 Oceania (client area) 884 Kyrgyzstan 2087 Brunei
    35 Nepal 896 Ireland 2094 Sri Lanka
    57 Thailand 901 Libya 2150 Panama
    73 India 904 Armenia 2175 Colombia
    144 Vietnam 921 Yemen 2273 Monaco
    192 France 926 Belarus 2343 Andorra
    207 United Kingdom 971 Luxembourg 2421 Turkmenistan
    208 Sweden 1036 New Zealand 2435 Laos
    209 Germany 1044 Japan 2488 East Timor
    213 Italy 1066 Pakistan 2490 Tonga
    214 Spain 1070 Malta 2588 Philippines
    386 United Arab Emirates 1091 Bahamas 2609 Venezuela
    391 Israel 1129 Argentina 2612 Bolivia
    397 Ukraine 1134 Bangladesh 2613 Brazil
    398 Russia 1158 Cambodia 2623 Costa Rica
    417 Kazakhstan 1159 Macao (China) 2626 Mexico
    428 Portugal 1176 Singapore 2639 Honduras
    443 Greece 1179 Maldives 2645 El Salvador
    471 Saudi Arabia 1180 Afghanistan 2647 Paraguay
    529 Denmark 1185 Fiji 2661 Peru
    565 Iran 1186 Mongolia 2728 Nicaragua
    578 Norway 1195 Indonesia 2734 Ecuador
    669 United States 1200 Hong Kong (China) 2768 Guatemala
    692 Syria 1233 Qatar 2999 Aruba
    704 Cyprus 1255 Iceland 3058 Ethiopia
    706 Czech 1289 Albania 3144 Bosnia and Herzegovina
    707 Switzerland 1353 Uzbekistan 3216 Dominican
    708 Iraq 1407 San Marino 3379 South Korea
    714 Netherlands 1416 Kuwait 3701 Malaysia
    717 Romania 1417 Montenegro 3839 Canada
    721 Lebanon 1493 Tajikistan 4450 Australia
    725 Hungary 1501 Bahrain 4460 Chinese mainland
    726 Georgia 1543 Chile -15 Asia - other
    731 Azerbaijan 1559 South Africa -14 South America - other
    734 Austria 1567 Egypt -13 North America - other
    736 Palestine 1590 Kenya -12 Europe - other
    737 Turkey 1592 Nigeria -11 Africa - other
    759 Lithuania 1598 Tanzania -10 Oceania - other
    763 Oman 1611 Madagascar -2 Outside the Chinese mainland - other

    ISPs outside the Chinese mainland

    ISP ID ISP
    -1 ISPs outside the Chinese mainland

    Notes

    The traffic/bandwidth data calculated based on the number of bytes recorded in the fifth field of an access log is different from the billable CDN traffic/bandwidth data for the following reason:

    • Only application-layer data can be recorded in access logs. The traffic generated during actual data transfer over the network is around 5 to 15% more than application-layer traffic, including the following two parts:
      • Consumption by TCP/IP headers: in TCP/IP-based HTTP requests, each packet has a maximum size of 1,500 bytes. This includes TCP and IP headers which account for 40 bytes and generate traffic during transfer but cannot be counted by the application layer. The overhead of this part is around 3%.
      • TCP retransmission: during normal data transfer over the network, around 3% to 10% of packets are lost on the internet and retransmitted by the server. This type of traffic cannot be counted by the application layer, which accounts for 3% to 7% of the total traffic.
    • As an industry standard, the billable traffic is the sum of the application-layer traffic and the overheads described above. Tencent Cloud CDN takes 10% as the overheads proportion, so the monitored traffic is around 110% of the logged traffic.

    Use Cases

    Sample Chinese mainland access log

    Sample access log outside the Chinese mainland