How to publish Kubernetes with External DNS, MetalLB and Traefik.

Kubernetes with External DNS, MetalLB and Traefik will help us to have web applications (in a microservice environment or not) be published, since the basic requirements are to resolve the name of the computer and the web path that leads to the DNS. application.

The big map

After the steps taken in K3s: Simplify Kubernetes and Helm v3 to deploy PowerDNS over Kubernetes we are going to shape a more complete Kubernetes solution so that you can publish services under your own domain and route and be accessible from outside. Always using the minimum resources in this task.

MetalLB

MetalLB will allow us to emulate the power of the load balancers of the Cloud providers, the requirements of this solution are Kubernetes version 1.13 or higher, and must be no other network balancer operating and that the network controller is supported in the list indicated in https://metallb.universe.tf/installation/network-addons/, we must bear in mind that K3s includes flannel that is supported and that in the case of others like Weave some modifications are required.

To install MetalLB you only need to apply the “yaml” file that deploy all the elements:

$ sudo kubectl apply -f "https://raw.githubusercontent.com/danderson/metallb/master/manifests/metallb.yaml"

And to activate MetalLB we must create a configuration (file pool.xml) that contains something like this:

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: my-ip-space
      protocol: layer2
      addresses:
      - 192.168.8.240/28

When applied with k3s kubectl apply -f pool.yml will configure MetalLB so that if there are services with “loadBalancer” they use one of the IPs defined in the specified range (in this case 192.168.9.240/28).

MetalLB give us a great advantage over other types of local solutions, since it does not require the use of SDN (such as Kubernetes on VMware NSX) or specific servers for publishing (such as OpenShift, which in addition to SDN, need specific machines to publish services).

Traefik

Traefik It is a router service with multiple features such as:

• Edge router
• Service Discovery
• Layer 7 load balancer
• TLS terminator and support Let’s Encrypt (ACME)
• It has a Kubernetes Ingress Controller
• It has an IngressRoute CRD
• Allows Canary Deployments
• Traces, metrics and registration

With K3s Traefik is automatically deployed when starting the master node, in the case of Kubernetes it will be done through Helm and for configuration we need a yaml file like the following:

dashboard:
  enabled: "true"
  domain: "traefik-dashboard.DOMINIO"
  auth:
    basic:
      admin: $apr1$zjjGWKW4$W2JIcu4m26WzOzzESDF0W/
rbac:
  enabled: "true"
ssl:
  enabled: "true"
  mtls:
    enabled: "true"
    optional: "true"
  generateTLS: "true"
kubernetes:
  ingressEndpoint:
    publishedService: "kube-system/traefik"
metrics:
  prometheus:
    enabled: "true"

This configuration is very similar to the one that K3s deploys and we have added the dashboard where we can check the status of the routes and services configured. There are several important points:

• rbac: Configure the permissions on Kubernetes necessary for its operation.
• dashboard: The Traefik status console requires that at least authentication be configured for access, in this case is basic, user: admin, password: admin.
• ssl: In the case of a production environment an SSL layer is configured with valid certificates, in our case being a test environment they will be self-generated, we also select the support mtls (mutual tls) for end-to-end security with the services that will be behind Traefik.
• kubernetes: Specific configuration to indicate what service the ingress will be published for, it can be a LoadBalancer or a NodePort. This is how Traefik becomes a router.

To execute the Chart with Helm 3 we must do:

cat <<EOF |helm install -f - -n kube-system traefik stable/traefik
dashboard:
  enabled: "true"
  domain: "traefik-dashboard.DOMINIO"
  auth:
    basic:
      admin: $apr1$zjjGWKW4$W2JIcu4m26WzOzzESDF0W/
rbac:
  enabled: "true"
ssl:
  enabled: "true"
  mtls:
    optional: "true"
  generateTLS: "true"
kubernetes:
  ingressEndpoint:
    publishedService: "kube-system/traefik"
metrics:
  prometheus:
    enabled: "true"
EOF

And as in our case the service is a LoadBalancer we verify that you have correctly taken the IP:

#kubectl -n kube-system get service -A
NAMESPACE     NAME                TYPE           CLUSTER-IP       EXTERNAL-IP     PORT(S)                                     AGE
...
kube-system   traefik             LoadBalancer   10.110.144.104   192.168.8.240   80:30026/TCP,443:30951/TCP,8080:31031/TCP   11s
kube-system   traefik-dashboard   ClusterIP      10.101.84.221    <none>          80/TCP                                      11s

Traefik’s capabilities are quite impressive, so we recommend reviewing your documentation at https://docs.traefik.io/

Kubernetes with External DNS

External-DNS It is a incubation feature that will allow us to configure a DNS server (in our case PowerDNS) so that it has DNS resources for the Ingress and the Services used. To install External-DNS we also use Helm

We assume that the external-dns namespace has already been created previously and that is where PowerDNS is deployed.

cat <<EOF |helm install -f - -n external-dns external-dns stable/external-dns
provider: "pdns"
pdns:
  apiUrl: "http://powerdns-service-api.external-dns.svc.DOMINIO"
  apiPort: 8081
  apiKey: "APIKEY"
txtOwnerId: "external-dns"
domainFilters:
  - "DOMINIO"
logLevel: "info"
interval: "10s"
rbac:
  create: "true"
EOF

APIKEY is the access key to the PowerDNS API configured in your deployment, apiUrl must point to the service that exposes that API and the “provider” configured is “pdns”.

Testing everything together

To see how Kubernetes with External DNS, PowerDNS, Traefik and MetalLB work together you can create the environment indicated in Kubernetes: Create a minimal environment for demos or in K3s: Simplify Kubernetes and apply Helm v3 to deploy PowerDNS over Kubernetes in addition to the previous steps, to verify access to External-DNS we must configure the IP obtained with kubectl -n external-dns get service powerdns-service-dns:

NAME                   TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE 
powerdns-service-dns   LoadBalancer   10.43.193.114   192.168.8.241   53:30581/UDP   6d3h

That we have to configure in the DNS client of the equipment (for example in /etc/resolv.conf) and we will be able to access the routes configured in traefik, such as the dashboard in https://traefik.DOMAIN/dashboard

And the PowerDNS administration service is accessible at https://powerdns-admin.DOMAIN and we will see all the resources published by external-dns that are those extracted from ingress and that will have the IP of Traefik’s LoadBalancer service.

After accessing, we proceed to deploy a new service like k8dash:

kubectl apply -f https://raw.githubusercontent.com/herbrandson/k8dash/master/kubernetes-k8dash.yaml

And to access it we create the following Ingress

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: k8dash
  namespace: kube-system
spec:
  rules:
  - host: k8dash.DOMINIO
    http:
      paths:
      - backend:
          serviceName: k8dash
          servicePort: 80
        path: /

When applying it we must find this new host in Traefik:

And we must have the ability to access the service with the URL since the DNS resolution must be correct: