The NGINX Ingress Controller maintained by the Kubernetes project is being retired. Since I currently use this controller, I need to migrate before updates stop.

It’s important to note that there are two NGINX Ingress Controllers:

• One by the Kubernetes project (now deprecated).
• One by the NGINX project.

While these controllers are very similar, they are not identical. You may notice discrepancies when following documentation, which can be confusing.

Choosing an ingress controller

In addition to the NGINX project’s controller, there are several other ingress controllers available as alternatives.

• It should be based on mature software that has been handling network traffic for years
• it should support Gateway API or have clear roadmap towards full Gateway API support
• It should be open source as much as possible without features behind a paywall
• Built-in observability through prometheus metrics
• I want an SSLLabs A+ rating like I had before

I want to use only the core Ingress and Gateway API features. Convenience features—such as built-in Let’s Encrypt integration or API gateway functionality—are not required. For example, I already use cert-manager for Let’s Encrypt integration. Avoiding vendor-specific features also helps prevent lock-in to a particular ingress controller.

As shown, NGINX and HAProxy have the most mature backends. Otherwise, the controllers appear comparable.Ultimately, the exact choice doesn’t matter much, since I only plan to use the controller for Ingress and Gateway API—not for any additional features they provide. Having had some issues in the past with the nginx controller from the ingress project. I decided to move to HA proxy.

HA proxy installation

helm repo add haproxytech https://haproxytech.github.io/helm-charts
helm repo update
helm upgrade --install haproxy haproxytech/kubernetes-ingress \
--version MYVERSION \
--create-namespace \
--namespace haproxy-external \
-f values.yaml

The most important part is the values.yaml

controller:
  config:
    response-set-header: |                                
      Strict-Transport-Security "max-age=31536000"     #A
      Server ssserver                                  #B 
    backend-config-snippet: |
      option forwarded                                 #C
  containerPort:
    http: 80                                           #C
    https: 443                                         #C
  allowPrivilegedPorts: true                           #C 


  ingressClass: external
  ingressClassResource:
    default: true
    name: external
  kind: DaemonSet                                     # D
  service:
    annotations:
      metallb.io/ip-allocated-from-pool: default
      metallb.io/loadBalancerIPs: 192.168.102.183     # E
    externalTrafficPolicy: Local                      # D
    type: LoadBalancer
  serviceMonitor:
    enabled: true                                     # F

• # A: By default HA proxy give an SSLLabs A rating. With this setting it becomes A+
• # B: Override server header to hide the type of backend service that is being used
• # C: Keycloak related configuration. This makes the internal service ports and internal ports be identical for HTTP and HTTPS which makes running keyclock in the backend easier. Since these ports are privileged ports the privileged ports must be allowed. Also, make sure that the standardized Forwarded header (RFC 7239) is used instead of the non-standard X-Forwarded* headers. This is so backend servers can find out the original requested host and port.
• # D: I want to see the original IP addresses on the controller. For this we need ExternalTrafficPolicy Local and use a DaemonSet to avoid dropping traffic. The load balancer assigns an IP to the service and terminates that IP on one of the kubernetes nodes. It then forwards traffic to a backend pod for that service running on that node when ExternalTrafficPolicy Local is used. If no such pod is running on the same node then traffic is dropped. The DaemonSet prevents this. When ExternalTrafficPolicy Cluster is ued, traffic can be forwarded to any backend pod on the cluster, losing the source IP in the process.
• # E: This is an annotation for metallb to configure the load balancer IP
• # F: Enable prometheus metrics.

Portability

ExternalName services

There was one portability issue that I encountered which is that when using an ExternalName service for an Ingress resource, the service ports must also be specified.

Forwarded and X-Forwarded headers

Also, Keycloak cost quite a lot of time. In the end it was because in my old setup I used both the Forwarded and X-Forwarded headers and I had two paths to keycloak: one for logins and another for the admin console.The admun  path relied on the Forwarded header and the login path worked also with the X-Forwarded headers. I have two paths because I have deployed two ingress classes with HA proxy (two helm installs), each one for a different ingress class. The Admin interface is hosted on another ingress class than the login interface. HA proxy apparently does not allow setting both the Forwarded and X-Forwarded headers at the same time.