tencent cloud

Feedback

Integration

Last updated: 2023-09-05 09:54:12

    Overview

    As a general solution to DNS optimization in the mobile internet era, HTTPDNS mainly addresses the following problems:
    Local DNS hijacking/failures
    Inaccurate local DNS scheduling
    The HTTPDNS SDK for Android mainly provides DNS and cache management capabilities based on HTTPDNS:
    When the SDK resolves a domain, it first uses HTTPDNS to get the DNS query result. In extreme cases, when HTTPDNS is unavailable, the result provided by local DNS will be used.
    The DNS query result returned by HTTPDNS will carry the TTL information, which the SDK will use to manage the cache storing the result provided by HTTPDNS.

    Preparations

    1. You have activated HTTPDNS as instructed in Activating HTTPDNS.
    2. You have added domains to be resolved in the HTTPDNS console as instructed in Adding a Domain.
    3. You have applied in the HTTPDNS console for SDK integration as instructed in SDK Activation Process.
    4. After activating the service, you have been assigned the configuration information such as authorization ID, AES and DES encryption keys, and HTTPS token. You can also view them on the Development Configuration page.
    
    Configuration information required for using the SDK for Android:
    Authorization ID: The unique ID of a development configuration used in HTTPDNS, namely, the dnsId parameter in the SDK used for DNS authentication.
    DES encryption key: The dnsKey parameter in the SDK, which you need to pass in when using the DES encryption method.
    AES encryption key: The dnsKey parameter in the SDK, which you need to pass in when using the AES encryption method.
    HTTPS encryption token: The token parameter in the SDK, which you need to pass in when using the HTTPS encryption method.
    Android APPID: The appkey of the SDK for Android used for authentication.

    SDK Integration

    HTTPDNS SDK integration

    Importing the AAR package

    2. Copy HttpDNSLibs\\HTTPDNS_ANDROID_xxxx.aar to the corresponding location in the application's libs folder.
    3. Add the following to build.gradle in the app module:
    Note
    In v4.3.0, local data storage is added. You also need to add a dependency on Room. For more information, see here.
    android {
    
    // ...
    
    repositories {
    flatDir {
    dirs 'libs'
    }
    }
    }
    
    dependencies {
    
    // ...
    
    implementation(name: 'HTTPDNS_Android_xxxx', ext: 'aar')
    // In v4.3.0, local data storage is added. You also need to add a dependency on Room.
    implementation "androidx.room:room-rxjava2:2.2.0"
    }

    Downloading the Maven repository

    1. Import dependencies via a POM file.
    <dependency>
    <groupId>io.github.dnspod</groupId>
    <artifactId>httpdns-sdk</artifactId>
    <version>4.4.0</version>
    <type>aar</type>
    </dependency>
    
    <!-- SDK for Tencent Cloud International -->
    <dependency>
    <groupId>io.github.dnspod</groupId>
    <artifactId>httpdns-sdk</artifactId>
    <version>4.4.0-intl</version>
    <type>aar</type>
    </dependency>
    Note
    HTTPDNS SDK v4.4.0 supports a version dedicated for Tencent Cloud International. If you want to use the version for Tencent Cloud International, obtain the initialization configuration from the HTTPDNS international console. See Access Documentation for details.

    Permission configuration

    <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE" />
    <uses-permission android:name="android.permission.ACCESS_WIFI_STATE" />
    <uses-permission android:name="android.permission.INTERNET" />
    
    <!-- Optional and used to get the mobile phone's IMEI for data reporting -->
    <uses-permission android:name="android.permission.READ_PHONE_STATE" />

    Network security configuration

    If targetSdkVersion of the application is 28 (Android 9.0) or later, HTTP network requests are not allowed by default. For more information, see Opt out of cleartext traffic. In this case, you need to add the IP used by the HTTPDNS request to the domain allowlist. The configuration is as follows:
    Configure in the AndroidManifest file.
    <?xml version="1.0" encoding="utf-8"?>
    <manifest ... >
    <application android:networkSecurityConfig="@xml/network_security_config"
    ... >
    ...
    </application>
    </manifest>
    Add the network_security_config.xml configuration file in the XML directory.
    <?xml version="1.0" encoding="utf-8"?>
    <network-security-config>
    <domain-config cleartextTrafficPermitted="true">
    <domain includeSubdomains="false">119.29.29.98</domain>
    <domain includeSubdomains="false">119.28.28.98</domain>
    </domain-config>
    </network-security-config>

    SDK initialization

    Initialization configuration (supported starting from v4.0.0)

    Note
    The HTTPDNS SDK provides multiple DNS optimization options, which can be combined for optimal DNS performance.
    You can configure setUseExpiredIpEnable(true) and setCachedIpEnable(true) to achieve optimistic DNS cache.
    The persistent cache feature is used to improve cache hit rate and shorten the time to the first meaningful paint. This feature stores the last DNS result locally and preferentially reads the local cache when an app is launched.
    If the cached DNS result has expired (TTL expired), since optimistic DNS cache allows the return of IP addresses in expired DNS results, the SDK returns the IP addresses in the expired DNS result (assuming that the IP addresses are available in most cases) and initiates an async request to update the cache.
    When optimistic DNS cache is enabled, cache cannot be hit (there is no cache) for the first DNS request of a domain, so 0;0 is returned, and an async DNS request is initiated to update the cache. Therefore, after enabling optimistic DNS cache, you need to ensure that local DNS will work if no cache is hit. For important domains, we recommend you enable DNS prefetch via preLookupDomains(String... domainList).
    If the server IP addresses are changed frequently, be sure to enable automatic cache update via persistentCacheDomains(String... domainList) and DNS prefetch via preLookupDomains(String... domainList) to ensure the accuracy of DNS results.
    Before getting the service instance, you can set some service attributes by initializing the configuration, which can be passed in as configuration items when the SDK is initialized.
    Note:
    [Disused from v4.4.0] The SDK log reporting capability is configured in the console. For more information, see here.
    DnsConfig dnsConfigBuilder = DnsConfig.Builder()
    // (Required) DNS ID, i.e., authorization ID, which can be applied for and obtained in the HTTPDNS console (https://console.tencentcloud.com/httpdns) for DNS authentication
    .dnsId("xxx")
    // (Required) DNS key, i.e., the key corresponding to the authorization ID, which can be obtained in Development Configuration (https://console.tencentcloud.com/httpdns/configure) for DNS authentication
    .dnsKey("xxx")
    // [Disused from v4.5.0] Internally scheduled by the SDK
    // (Required) Use `119.29.29.98` (default value) if `channel` is `desHttp()` or `aesHttp()`. Use `119.29.29.99` if `channel` is `https()`
    .dnsIp("xxx")
    // (Optional) Channel configuration: The default value is `desHttp()` (DES encryption based on HTTP request). Valid values also include `aesHttp()` and `https()`. (Note that only when you select the HTTPS channel, you need to select the `dnsip` of `119.29.29.99` and pass in the `token`, for example, `.dnsIp('119.29.29.99').https().token('....')`)
    .desHttp()
    // (Optional and needs to be set when the HTTPS channel is selected) Token, which can be applied for and obtained in the HTTPDNS console (https://console.tencentcloud.com/httpdns) for HTTPS verification. This is required only when `https()` is selected
    .token("xxx")
    // (Optional) Log granularity. If DEBUG printing is enabled, pass in "Log.VERBOSE"
    .logLevel(Log.VERBOSE)
    // (Optional) Domains for DNS prefetch, in the format of `"baidu.com", "qq.com"`. You can configure up to 10 domains. DNS prefetch is triggered only during initialization.
    .preLookupDomains("baidu.com", "qq.com")
    (Optional) Domains for automatic cache update, in the format of `"baidu.com", "qq.com"`. For the configured domains, the SDK will initiate DNS requests automatically to update their cache when 75% of the TTL has elapsed, so that cache will always be hit for these domains. You can configure up to 10 domains. Configure the domains for automatic cache update and those for DNS prefetch separately.
    .persistentCacheDomains("baidu.com", "qq.com")
    // (Optional) Items for IP address ranking, configured as a list of `IpRankItem(hostname, port)`, where port is optional and defaults to 8080. An example value is `IpRankItem("qq.com", 443)`. The SDK will perform a socket connection speed test for the configured items and rank the resolved IP addresses. IP address ranking does not block the current DNS query and takes effect at the next query. You can configure up to 10 items.
    .ipRankItems(ipRankItemList)
    // (Optional) Manually specify the support of network stack. Pass in 1 for IPv4 resolution only, pass in 2 for IPv6 resolution only, and pass in 3 for IPv4/IPv6 dual stack resolution. By default, a DNS request will be initiated according to the support of the client's local network stack
    .setCustomNetStack(3)
    // (Optional) Set whether to allow the use of expired cache. The default value is `false`. When the value is `false`, the SDK preferentially obtains the unexpired cache result. If this cannot be obtained, it will wait for the DNS request to complete and then return the result.
    // (Optional) Set whether to allow the use of expired cache. When this parameter is set to `true`, the SDK directly returns the cached DNS result or `0;0` if there is no cached result, and initiates an async DNS request to update the cache if there is no cached result or the cache has expired. Since the async APIs (getAddrByNameAsync and getAddrsByNameAsync) always return the unexpired DNS result in the callback, the async APIs are unavailable when this parameter is set to `true`. In this case, we recommend you use the sync APIs (getAddrByName and getAddrsByName). You can use `localdns (InetAddress)` to return the result if HTTPDNS fails to return a result.
    .setUseExpiredIpEnable(true)
    // (Optional) Set whether to enable the local cache (Room). The default value is `false`.
    .setCachedIpEnable(true)
    // (Optional) Set the DNS request timeout period, which is 2000 ms by default
    .timeoutMills(2000)
    // (Optional) The ECS (EDNS-Client-Subnet) value of the DNS request. By default, the HTTPDNS server will query the client's egress IP in order to query the IP for the DNS split zone. You can specify the split zone's IP address by passing in an IPv4/IPv6 address
    .routeIp("XXX")
    // (Optional) [Disused from v4.4.0] The SDK log reporting capability is configured in the console.
    .enableReport(true)
    // End with `build()`
    .build();
    
    MSDKDnsResolver.getInstance().init(this, dnsConfigBuilder);

    Initialization for earlier versions (upgrade is recommended)

    The service addresses for the HTTP and HTTPS protocols are 119.29.29.98 and 119.29.29.99 respectively (use the 119.29.29.99 IP only when you select an encryption method on your own and channel is Https).
    The new version of the APIs now can be accessed at 119.29.29.99/98, and the original HTTPDNS service address 119.29.29.29 is for development and debugging purposes only without an SLA guarantee, so it is not recommended for business purposes. Migrate your business to 119.29.29.99/98 as soon as possible.
    The IP is as provided in API Description.
    When you integrate HTTPDNS through the SDK, if HTTPDNS does not find a DNS query result, the domain will be resolved by local DNS, and the result provided by local DNS will be returned.

    SDK integration methods

    There are two methods to integrate the DNS capability of the HTTPDNS SDK into the HTTP (HTTPS) network access process:
    Method 1: URL replacement Replace the host part in the URL to get a new URL for accessing the network. In this method, the URL discards the domain information; therefore, for network requests that require domain information, there is a lot of work to do to ensure compatibility.
    Method 2: DNS replacement Incorporate the DNS capability of HTTPDNS in the network access process to replace local DNS in the original process.
    In this method, you don't need to modify the URLs of the requests one by one, so there is no need to conduct extra work to ensure compatibility, but the network library used on the business side must support DNS replacement.
    DNS replacement can be implemented by hooking the system's DNS function, but the function is already used in the HTTPDNS SDK, so hooking the function again may cause recursive calls or even stack overflow.
    For more information on how to connect to different network libraries, see the corresponding connection documents (in the current directory) and the samples (in the HttpDnsSample directory).

    Compatibility for URL replacement

    As described above, for network requests that require domain information (usually when multiple domains are mapped to the same IP), you need to conduct extra work to ensure compatibility. The following describes how to ensure compatibility from the perspective of protocols, and the specific implementation method is subject to the implementation of the network library.
    HTTP compatibility For HTTP requests, you need to notify the server of the domain information by specifying the host field in the header. For more information on the host field, see Host.
    HTTPS compatibility
    HTTPS is a version of HTTP over TLS; therefore, for HTTPS requests, you also need to set the host field.
    In HTTPS requests, you should perform a TLS handshake first. During the TLS handshake, the server will send its own digital certificate to you for identity verification; therefore, you also need to notify the server of the domain information during the TLS handshake. In the TLS protocol, you can specify the domain information by using the SNI extension as detailed in Server Name Indication.

    Using HTTP proxy locally

    Note
    If an HTTP proxy is used locally, we recommend you not use HTTPDNS to resolve domains.
    The following provides a detailed analysis for both integration methods:
    URL replacement According to the HTTP/1.1 protocol, when an HTTP proxy is used, the request line will carry complete server address information on the client. For more information, see origin-form. In this case (i.e., an HTTP proxy is used locally, the business has been connected to the HTTPDNS SDK by replacing the URL, and the host field has been set correctly), the HTTP request received by the HTTP proxy will contain the server IP information (in the request line) and domain information (in the host field). However, how the HTTP proxy will send the HTTP request to the real destination server depends on the proxy implementation, and the proxy may directly discard the set host field, causing the network request to fail.
    DNS replacement Taking the OkHttp network library as an example, after the HTTP proxy is enabled locally, OkHttp will resolve only the host of the configured HTTP proxy but not the host field in the HTTP request. In this case, it is useless to enable HTTPDNS.
    Check whether an HTTP proxy is used locally with the following code:
    val host = System.getProperty("http.proxyHost")
    val port = System.getProperty("http.proxyPort")
    if (null != host && null != port) {
    // An HTTP proxy is used locally
    }

    Integration verification

    Log verification

    When you pass in true for the debug parameter in the init API to filter logs with the HTTPDNS tag, if you see logs related to the local DNS (ldns_ip) and HTTPDNS (hdns_ip), the connection is correct.
    The ldns_ip key indicates the DNS query results provided by the local DNS.
    The hdns_ip key indicates the DNS query results of A records provided by HTTPDNS.
    The hdns_4a_ips key indicates the DNS query results of AAAA records provided by HTTPDNS.
    The a_ips key indicates the set of IPv4 addresses returned by the DNS API.
    The 4a_ips key indicates the set of IPv6 addresses returned by the DNS API.

    Simulating local DNS hijacking

    When you simulate hijacking local DNS, if the application works properly, HTTPDNS has been integrated successfully.
    Note
    Due to the caching mechanism of local DNS, when simulating local DNS for integration verification, make sure that the cache of local DNS has been cleared. You can clear the cache by restarting the server or switching the network. During verification, be sure to compare the effects of local DNS and HTTPDNS.
    Modify the hosts file on your server.
    Local DNS gets the DNS query result by reading the hosts file on your server first.
    You can simulate hijacking local DNS by modifying the hosts file to point the domain to an incorrect IP.
    You can directly modify the hosts file on a root server.
    Modify the DNS server configuration.
    You can simulate hijacking local DNS by modifying the configuration of the DNS server to point it to an unavailable IP (such as another IP on the LAN).
    When your server is connected to Wi-Fi, you can modify the DNS server settings by changing the IP settings to Static in the Advanced Settings option of the Wi-Fi (this operation may vary slightly by server).
    Use a DNS modifier to modify the DNS server configuration (usually by tampering with the DNS packet through VPN).

    Packet capture verification

    The following takes accessing a network over HTTP as an example:
    Note
    Common mobile HTTP/HTTPS packet capture tools (such as Charles and Fiddler) capture packets through an HTTP proxy, so they are not suitable for verifying whether HTTPDNS works. For more information, see Using the HTTP proxy locally.
    Use tcpdump to capture packets.
    On a root server, you can use the tcpdump command to capture packets.
    On a non-root server, the system may have a built-in debugging tool that can be used to get the packet capture result (the enabling method varies by server).
    Observe the packet capture result with WireShark.
    For HTTP requests, you can observe plaintext information and confirm whether the IP used during request sending is the one returned by the SDK by comparing the log and the specific packet capture record. The details are as shown below:
    
    From the perspective of packet capture, the request for xw.qq.com is eventually sent to the server at the IP address 183.3.226.35.
    For HTTPS requests, a TLS handshake packet is actually a plaintext packet. After setting the SNI extension (see HTTPS compatibility), you can verify whether the IP used during request sending is the one returned by the SDK by comparing the log and the specific packet capture record. The details are as shown below:
    
    From the perspective of packet capture, the request for xw.qq.com is eventually sent to the server at the IP address 183.3.226.35.

    Notes

    getAddrByName is a time-consuming sync API, so it should be called on a child thread.
    If the business on the client is bound to the host, such as the HTTP service of the host or the CDN service, you need to specify the host field of the HTTP header after replacing the domain in the URL with the IP returned by HTTPDNS.
    Take URLConnection as an example:
    URL oldUrl = new URL(url);
    URLConnection connection = oldUrl.openConnection();
    // Get the HTTPDNS query result
    String ips = MSDKDnsResolver.getInstance().getAddrByName(oldUrl.getHost());
    String[] ipArr = ips.split(";");
    if (2 == ipArr.length && !"0".equals(ipArr[0])) { // Get the IP successfully through HTTPDNS, replace the URL, and set the host header
    String ip = ipArr[0];
    String newUrl = url.replaceFirst(oldUrl.getHost(), ip);
    connection = (HttpURLConnection) new URL(newUrl).openConnection(); // Set the host field of the HTTP request header
    connection.setRequestProperty("Host", oldUrl.getHost());
    }
    Taking curl as an example, if you want to access www.qq.com, and the IP obtained by HTTPDNS is 192.168.0.111, then you can access it as follows:
    curl -H "Host:www.qq.com" http://192.168.0.111/aaa.txt
    Check whether an HTTP proxy is used locally. If an HTTP proxy is used, we recommend you not use HTTPDNS to resolve domains. Example:
    String host = System.getProperty("http.proxyHost");
    String port= System.getProperty("http.proxyPort");
    if (null != host && null != port) {
    // Local proxy mode is used
    }
    
    Contact Us

    Contact our sales team or business advisors to help your business.

    Technical Support

    Open a ticket if you're looking for further assistance. Our Ticket is 7x24 avaliable.

    7x24 Phone Support