Configure Nginx Ingress Controller for TLS termination on Kubernetes on Azure


If we need TLS termination on Kubernetes, you can use ingress controller. On Azure, you can use Nginx Ingress controller. (Now, Microsoft working with Azrue ingress controller which uses Application gateway)

see Status of Kubernetes on Azure

Nginx Ingress works

I'd like to share how to configure Nginx Ingress Controller on Kubernetes on Azure.

Clone Nginx Ingress repository

You can find ingress repo in here. Please clone it.

$ git clone https://github.com/kubernetes/ingress.git

You can find a lot of instruction and YAML file to configure Nginx Ingress.

Deploying the Nginx Ingress controller

Just follow this instruction.

Deploy default-backend pod.

$ cd ingress/examples/deployment/nginx
 $ kubectl apply -f default-backend.yaml
 deployment "default-http-backend" created
 service "default-http-backend" created
 $ kubectl -n kube-system get po
 NAME READY STATUS RESTARTS A GE
 default-http-backend-2657704409-tztz5 1/1 Running 0 1 
      :

Then deploy ingress controller. It is deployed on the kube-system name space.

$ kubectl apply -f nginx-ingress-controller.yaml
 deployment "nginx-ingress-controller" created
 $ kubectl -n kube-system get po
 NAME READY STATUS RESTARTS A GE
 default-http-backend-2657704409-tztz5 1/1 Running 0 5 0s
          :
 nginx-ingress-controller-3752011415-x2dks 0/1 Running 0 1 5s

TLS certificates

Reading this page, create a TLS self-signed certificate for testing.

I execute openssl command via Bash on Ubuntu on Windows. When I tried on GitBash it didn't work.

$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/O=nginxsvc"
 Generating a 2048 bit RSA private key
 .............................................+++
 ...........+++
 writing new private key to 'tls.key'
 -----

then create secret on Kubernetes using the self signed certificate.

$ kubectl create secret tls tls-secret --key tls.key --cert tls.crt
 secret "tls-secret" created

Deploy Test HTTP Service

Following this instruction. Let's deploy a sample web application.

$ kubectl create -f http-svc.yaml
 service "http-svc" created
 replicationcontroller "http-svc" created
 $ kubectl get po
 NAME READY STATUS RESTARTS AGE
 http-svc-23pm5 1/1 Running 0 4m
 http-svc-92zfc 1/1 Running 0 4m
 $ kubectl get service
 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
 http-svc 10.0.70.205 <nodes> 80:30301/TCP 1h
 kubernetes 10.0.0.1 <none> 443/TCP 14d

Configure TLS termination

Follow this instruction. Go ingress/examples/tls-termination/nginx/ directory.

$ kubectl create -f nginx-tls-ingress.yaml

nginx-tls-ingress.yaml

apiVersion: extensions/v1beta1
 kind: Ingress
 metadata:
  name: nginx-test
  annotations:
  kubernetes.io/ingress.class: "nginx"
 spec:
  tls:
  # This assumes tls-secret exists.
  - secretName: tls-secret
  rules:
  - http:
  paths:
  - backend:
  # This assumes http-svc exists and routes to healthy endpoints.
  serviceName: http-svc
  servicePort: 80

 

However it still not enough for Azure environment.

You need to expose the Ingress Replica set. You can find the ingress replica set name.

$ kubectl get rs --namespace kube-system
 NAME DESIRED CURRENT READY AGE
 default-http-backend-2657704409 1 1 1 29m
 heapster-v1.2.0-1448994189 1 1 1 14d
 kubernetes-dashboard-696481038 1 1 1 14d
 nginx-ingress-controller-3752011415 1 1 1 28m

Then expose it. specify the Replica Set name and expose 443 for SSL. You need to "--type=LoadBalancer" to expose IP. Also you need to specify "--namespace kube-system'. Nginx ingress controller is deployed with kube-system namespace.

$ kubectl expose rs nginx-ingress-controller-3752011415 --port=443 --target-p ort=443 --name=nginx-ingress-ssl --type=LoadBalancer --namespace kube-system
 service "nginx-ingress-ssl" exposed
 $ kubectl get services --namespace kube-system -w
 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
 default-http-backend 10.0.244.231 <none> 80/TCP 34m
 heapster 10.0.111.25 <none> 80/TCP 14d
 kube-dns 10.0.0.10 <none> 53/UDP,53/TCP 14d
 kubernetes-dashboard 10.0.106.144 <nodes> 80:30177/TCP 14d
 nginx-ingress-ssl 10.0.232.84 <pending> 443:32541/TCP 46s
 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
 nginx-ingress-ssl 10.0.232.84 13.71.145.137 443:32541/TCP 1m

Now, external IP has been exposed.  Let's test this.

$ curl https://13.71.145.137
  % Total % Received % Xferd Average Speed Time Time Time Curre nt
  Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
 curl: (60) SSL certificate problem: self signed certificate
 More details here: https://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
  of Certificate Authority (CA) public keys (CA certs). If the default
  bundle file isn't adequate, you can specify an alternate file
  using the --cacert option.
 If this HTTPS server uses a certificate signed by a CA represented in
  the bundle, the certificate verification probably failed due to a
  problem with the certificate (it might be expired, or the name might
  not match the domain name in the URL).
 If you'd like to turn off curl's verification of the certificate, use
  the -k (or --insecure) option.

$ curl https://13.71.145.137 -k
  % Total % Received % Xferd Average Speed Time Time Time Curre nt
  Dload Upload Total Spent Left Speed
  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 100 430 0 430 0 0 1530 0 --:--:-- --:--:-- --:--:-- 162 2CLIENT VALUES:
 client_address=10.244.1.25
 command=GET
 real path=/
 query=nil
 request_version=1.1
 request_uri=http://13.71.145.137:8080/

SERVER VALUES:
 server_version=nginx: 1.9.11 - lua: 10001

HEADERS RECEIVED:
 accept=*/*
 connection=close
 host=13.71.145.137
 user-agent=curl/7.50.3
 x-forwarded-for=127.0.0.1
 x-forwarded-host=13.71.145.137
 x-forwarded-port=443
 x-forwarded-proto=https
 x-real-ip=127.0.0.1
 BODY:
 -no body in request-

 

Done! Enjoy Nginx Ingress Controller on Azure.


Comments (7)

  1. Markus Padourek says:

    Following this step-by-step on latest master (23.05.2017) leads to an error ‘curl: (35) gnutls_handshake() failed: The TLS connection was non-properly terminated.’. However if I checkout a commit around the 28.02.2017, e.g. d3b952552a8152e84905c04054387c38dd8ac943 that is based on ‘nginx-ingress-controller:0.9.0-beta.2’ it all works as shown here. Not sure what exactly the cause is though.

    1. Markus Padourek says:

      Ok, I checked it out and the example here still works with ‘nginx-ingress-controller:0.9.0-beta.3’ but not anymore with ‘nginx-ingress-controller:0.9.0-beta.4’. I opened up an Issue about it: https://github.com/kubernetes/ingress/issues/758

  2. Sagar Borse says:

    I followed all the steps except for “kubectl expose ….” command, as i am not using Azure. I am using on-premise 3 node cluster.
    When i curl to the machine with http it works, but it fails with https. Can you please give me some pointer or clue where i might be going wrong.
    Note : I am using a self signed certificate generated using openssl command line utility.

  3. Vikram says:

    Hi, First of all thanks for providing step by step details and making life easier.

    I get this at the end of the step curl https://13.82.91.62 -k – Unauthorized

    I also see that the external ip is pending in svc

    kubectl get services –namespace kube-system -w
    NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
    default-http-backend 10.254.25.89 80/TCP 7m
    nginx-ingress-ssl 10.254.227.3 443:30541/TCP 13s

    [testadmin@hostname nginx]$ kubectl describe svc nginx-ingress-ssl –namespace kube-system
    Name: nginx-ingress-ssl
    Namespace: kube-system
    Labels: k8s-app=nginx-ingress-controller
    pod-template-hash=1449474453
    Selector: k8s-app=nginx-ingress-controller,pod-template-hash=1449474453
    Type: LoadBalancer
    IP: 10.254.227.3
    Port: 443/TCP
    NodePort: 30541/TCP
    Endpoints: 172.16.59.6:443
    Session Affinity: None
    No events.

    Could you let me know why this happens?

    Thanks,

    1. Vikram says:

      Just checked in nginx logs

      I0718 21:21:10.526664 7 controller.go:438] ingress backend successfully reloaded…
      I0718 21:21:24.448413 7 status.go:310] updating Ingress default/nginx-test status to [{13.82.93.59 }]

      and curl https://13.82.93.59 -k worked actually. earlier i was curling the masters ip.

      I have one question, I guess this setup does not provide HA or fault tolerance to Load balancing, just basic redirection, SSL termination. Do you know how I can create a HA application using load balancer such that even if one node goes down, the application could be accessed using the same url?

  4. egooge says:

    Great tutorial! It saved me a lot of time!

    1. egooge says:

      I did run into a problem with certificates that was solved by using the ingress rule pasted below.
      This was explained by pieterlang here: https://github.com/kubernetes/ingress/issues/1044

      spec:
      rules:
      – host: my.cn.domain
      http:
      paths:
      – backend:
      serviceName: my-https-svc
      servicePort: 80
      path: /
      tls:
      – hosts:
      – my.cn.domain
      secretName: my-cn-secret

Skip to main content