Strip suffix from Host header in k8s nginx ingress?

The short version:

This is something we do with HAProxy. What is the equivalent for nginx when used as an kubernetes ingress?

       acl is_extra hdr_end(host) .extra
       http-request replace-header Host (.*)\.extra$ \1 if is_extra

The long version:

How can we strip a suffix from the host header in our HTTP requests?

In an HTTP request, the Host header matches *.example.com. For example:

• foo.example.com
• bar.example.com

However we also need to be able to receive hostnames with a .extra suffix:

• foo.example.com.extra
• bar.example.com.extra

The legacy apps can't understand the .extra notation, so we need to rewrite the Host header and strip the .extra suffix.

If there were only 2 such hosts, we could simply make 2 rewrite rules, one for foo and one for bar.

Something like:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: foo-extra
  annotations:
    nginx.ingress.kubernetes.io/upstream-vhost: foo.example.com
spec:
  ingressClassName: nginx-internal
  rules:
  - host: "foo.example.com.extra"
    http:
      ...
      ...
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: bar-extra
  annotations:
    nginx.ingress.kubernetes.io/upstream-vhost: bar.example.com
spec:
  ingressClassName: nginx-internal
  rules:
  - host: "bar.example.com.extra"
    http:
      ...
      ...

However the list of hosts is very large and changes frequently. Therefore, we need something dynamic.

For example, something like (fictional)

    - host: "*.example.com.extra"
      fictional-rewrite-host-with-regex: "$1.example.com"

Or even "something that magically strips the last 6 chars:

    - host: "*.example.com.extra"
      fictional-strip-last-chars-from-header: "6"

For a stand-alone instance of NGINX, the way you would do this is to create a map in the http section of the configuration, which will allow you to create new variables from existing values. To change the host header, you would use the $http_host variable. You can then use a regex to capture part of the existing host value and write it to a new variable.

map $http_host $modified_host {
    "~^(.*)\.extra$" "$1";
    default $http_host;
}

Then this new variable can be used in the location section of the configuration to overwrite the host header using the proxy_set_header directive.

location / {
    proxy_set_header Host $modified_host;
}

So the full configuration could look something like this.

...

http {
    ...

    map $http_host $modified_host {
        "~^(.*)\.extra$" "$1";
        default $http_host;
    }
    
    server {
        ...

        location / {
            proxy_set_header Host $modified_host;
        }
    }
}

From there we just need to look at how to transpose these configuration values into ingress-nginx so the controller will build the configuration that we want.

Since map/variables are global they will have to be pushed as configuration to the controller itself. This can be done as an http-snippet in the ConfigMap of the controller. Since I'm using Helm to configure the controller I can pass it in via a values file that looks like this.

controller:
  config:
    http-snippet: |
      map $http_host $modified_host {
        "~^(.*)\.extra$" "$1";
        default $http_host;
      }

Once this is deployed, the new variable can be consumed using an annotation on the ingress object. The nginx.ingress.kubernetes.io/upstream-vhost annotation corresponds to the proxy_set_header directive in the native NGINX config. So the ingress object would look something like this.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: extra-example
  annotations:
    nginx.ingress.kubernetes.io/upstream-vhost: $modified_host
spec:
  ingressClassName: ingress-nginx
  rules:
  - host: "bar.example.com.extra"
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: extra-example
            port:
              number: 80

A negative aspect to this approach is that you must update the controller as well as the ingress object. In some cases, this might need to be done by two different people or teams, depending on cluster permissions. One useful thing I found in testing, is that the validation webhook will reject the ingress object if it tries to use a variable that hasn't been deployed to the controller yet. So you can be confident, if the ingress deploys successfully, that the controller configuration has also been updated.