FAQs about Statistical Analysis
Last updated: 2019-08-12 19:05:09PDF
- How is the bandwidth statistics in access monitoring collected?
- Why is the traffic in the monitoring information different from that in the log?
- How to calculate the traffic hit rate?
- How to fix the problem with low traffic hit rate?
- Do status code statistics include all the status codes?
- How district and ISP statistics are calculated?
How is the bandwidth statistics in access monitoring collected?
Each CDN node collects traffic data in real time and reports it to the computing center which aggregates the data into total traffic data and display the bandwidth statistics by dividing the total traffic by the duration of use.
- If the total traffic generated in a minute is 6 MB, then the corresponding bandwidth is (6 * 8) / 60 = 0.8 Mbps.
- As the usage for bill-by-bandwidth is calculated based on the statistics with a 5-minute granularity, the corresponding bandwidth value is total traffic in 5 minutes / 300 seconds.
Why is the traffic in the monitoring information different from that in the log?
The traffic counted based on the downstream bytes in the log of an accelerated domain name is only about the data at the application layer, while the traffic generated during actual data transfers over the network is around 5% to 15% more than application-layer traffic.
- Consumption by TCP/IP headers: In TCP/IP-based HTTP requests, each packet has a maximum size of 1,500 bytes, including TCP and IP headers of 40 bytes, which generate traffic during transfer but cannot be counted by the application layer. The overheads of this part is around 3%.
- TCP retransmission: During normal data transfer over the network, around 3% to 10% packets are lost on the internet, and the server will retransmit the lost ones. This type of traffic cannot be counted by the application layer, which accounts for 3% to 7% of the total traffic.
As an industry standard, the billable traffic is the sum of the application-layer traffic and the overheads as described above. Tencent Cloud CDN takes 10% as the overheads proportion, so the monitored traffic is around 110% of the logged traffic.
How to calculate the traffic hit rate?
By default, CDN enables L2 cache (edge layer and intermediate layer). As long as a request hits either layer for response, it is counted as a CDN node hit.
Traffic hit rate = (total downstream traffic - origin-pull traffic) / total downstream traffic.
How to fix the problem with low traffic hit rate?
- Check whether the cache is purged: Cache purge clears the specified content on the node, leading to temporarily low traffic hit rate.
- Check whether new resources are put into the origin server: High amounts of new resources may cause origin-pulls by CDN, leading to low traffic hit rate.
- Check whether the origin server works properly: If it is malfunctioning with a lot of 4XX or 5XX error, the traffic hit rate will be affected.
- Check whether the cache expiration policy is correctly configured: View the Cache Expiration Configuration section on the Cache Configuration page in the console. The cache expiration policies are displayed in ascending order in terms of priority, i.e., a policy takes precedence over the one above it.
- Check whether Range GETs is enabled: Check the Range GETs section on the Origin-pull Configuration page in the console. If it is disabled, files will be pulled in their entirety instead of multiple parts as requested during origin-pull, which increases the origin-pull traffic and lowers the hit rate.
- Check whether Ignore Query String is enabled: View the Ignore Query String section on the Access Configuration page in the console. If it is disabled, caching will be performed based on the full path. In this case, if the same resource are requested by different parameters, it cannot be matched and will cached multiple times, which lowers the traffic hit rate.
Do status code statistics include all the status codes?
Yes. In the new version of CDN statistical analysis, monitoring curves are drawn for all status codes generated by origin servers, making it easier for you to troubleshoot.
How district and ISP statistics are calculated?
The district and ISP statistics are calculated based on the client IPs in the access log. As the calculation is completed based on the log, the simply accumulated billable data varies from the billable data when all districts or all ISPs is selected. For details, see the question #2 above.