Implementing Custom Domain Name Resolution in TKE

Last updated: 2021-01-11 10:33:35

    Overview

    When using TKE or EKS, you need to resolve the custom internal domain names in the following scenarios:

    • You build an external centralized storage service, and need to send the monitoring or log collection data in the cluster to the external storage service through a fixed internal domain name.
    • During the containerization of traditional services, the code of some services is configured to call other internal services with a fixed domain name, and the configuration cannot be modified, that is, the Service name of Kubernetes cannot be used for calling.

    Solutions

    This document describes the following three solutions for using custom domain name resolution in a cluster:

    Solutions Advantages
    Solution 1: Using CoreDNS Hosts plugin to configure arbitrary domain name resolution This solution is simple and intuitive. You can add arbitrary resolution records.
    Solution 2: Using CoreDNS Rewrite plugin to map a domain name to the service in the cluster There is no need to know the IP address of the resolution record in advance, but the IP address mapped by the resolution record must be deployed in the cluster.
    Solution 3: Using CoreDNS Forward plugin to set self-built DNS as the upstream DNS You can manage a large number of resolution records. As all records are managed in the self-built DNS, you do not need to modify the CoreDNS configuration when adding or deleting records.

    Note:

    In solution 1 and 2, you need to modify CoreDNS configuration file each time you add a resolution record. Please select the solution based on your actual needs.

    Examples

    Solution 1: Using CoreDNS Hosts plugin to configure arbitrary domain name resolution

    1. Run the following command to modify the configmap of CoreDNS, as shown below:

      kubectl edit configmap coredns -n kube-system
    2. Modify the hosts configuration to add the domain name to the hosts, as shown below:

      hosts {
           192.168.1.6     harbor.oa.com
           192.168.1.8     es.oa.com
           fallthrough
      }

      Note:

      Map harbor.oa.com to 192.168.1.6 and map es.oa.com to 192.168.1.8.

      The complete configurations are as follows:

      apiVersion: v1
      data:
         Corefile: |2-
           .:53 {
               errors
               health
               kubernetes cluster.local. in-addr.arpa ip6.arpa {
                   pods insecure
                   upstream
                   fallthrough in-addr.arpa ip6.arpa
               }
               hosts {
                   192.168.1.6     harbor.oa.com
                   192.168.1.8     es.oa.com
                   fallthrough
               }
               prometheus :9153
               forward . /etc/resolv.conf
               cache 30
               reload
               loadbalance
           }
      kind: ConfigMap
      metadata:
         labels:
           addonmanager.kubernetes.io/mode: EnsureExists
         name: coredns
         namespace: kube-system

    Solution 2: Using CoreDNS Rewrite plugin to map a domain name to the service in the cluster

    If you need to deploy a service with a custom domain name in a cluster, you can use the Rewrite plugin of CoreDNS to specify the domain name to the ClusterIP of a Service.

    1. Run the following command to modify the configmap of CoreDNS, as shown below:

      kubectl edit configmap coredns -n kube-system
    2. Run the following command to add the Rewrite configuration, as shown below:

      rewrite name es.oa.com es.logging.svc.cluster.local

      Note:

      Map the es.oa.com to the es service deployed under the logging namespace. Separate multiple domain names with carriage returns.

      The complete configurations are as follows:

      apiVersion: v1
      data:
         Corefile: |2-
           .:53 {
               errors
               health
               kubernetes cluster.local. in-addr.arpa ip6.arpa {
                   pods insecure
                   upstream
                   fallthrough in-addr.arpa ip6.arpa
               }
               rewrite name es.oa.com es.logging.svc.cluster.local
               prometheus :9153
               forward . /etc/resolv.conf
               cache 30
               reload
               loadbalance
           }
      kind: ConfigMap
      metadata:
         labels:
           addonmanager.kubernetes.io/mode: EnsureExists
         name: coredns
         namespace: kube-system

    Solution 3: Using CoreDNS Forward plugin to set self-built DNS as the upstream DNS

    1. Check the forward configuration. The default configuration of forward is as follows, which means that the domain name that is not in the cluster is resolved by the nameserver configured in the /etc/resolv.conf file of the node where CoreDNS is located.
      forward . /etc/resolv.conf
    2. Configure forward and replace /etc/resolv.conf with the self-built DNS server address, as shown below:
      forward . 10.10.10.10
      The complete configurations are as follows:
      apiVersion: v1
      data:
         Corefile: |2-
           .:53 {
               errors
               health
               kubernetes cluster.local. in-addr.arpa ip6.arpa {
                   pods insecure
                   upstream
                   fallthrough in-addr.arpa ip6.arpa
               }
               prometheus :9153
               forward . 10.10.10.10
               cache 30
               reload
               loadbalance
           }
      kind: ConfigMap
      metadata:
         labels:
           addonmanager.kubernetes.io/mode: EnsureExists
         name: coredns
         namespace: kube-system
    3. Configure the resolution record of the custom domain name to the self-built DNS. It is recommended to add the nameserver in /etc/resolv.conf on the node to the upstream of self-built DNS, because some services rely on Tencent Cloud internal DNS resolution. If it is not set as the upstream of self-built DNS, some services may fail to work properly. This document takes BIND 9 as an example to modify the configuration file and write the upstream DNS address into forwarders, as shown below:
      options {
           forwarders {
                   183.60.83.19;
                   183.60.82.98;
           };
           ...

    References

    Was this page helpful?

    Was this page helpful?

    • Not at all
    • Not very helpful
    • Somewhat helpful
    • Very helpful
    • Extremely helpful
    Send Feedback
    Help