EKS ingress-nginx and NLB with https redirect

I'm having issues with the nlb lately, it was quite an adventure to have nlb with https termination on the lb working with a redirection http=>https and an ingress-nginx on EKS.

Now, I want to have the X-Forwarded headers passed to the pod, but that breaks the http=>https redirection, I get a 400 on http requests.

On the service, I tried to put the service with http or tcp protocol, same thing.

Adding the service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*" header to the service, activates the proxy protocol v2 on all targets, and activating use-proxy-protocol: 'true' in the configmap for nginx breaks the http-snippet with the 308 redirection:

http-snippet: |
    server {
      listen 2443;
      return 308 https://$host$request_uri;
    }

Does anyone has a way to make it so that it can use the nlb with all the good header and the redirect working?

EDIT at comment request adding full working config

apiVersion: v1
kind: ConfigMap
metadata:
  labels:
    app.kubernetes.io/version: 0.41.0
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller
data:
  http-snippet: |
    server {
      listen 2443 proxy_protocol;
      return 308 https://$host$request_uri;
    }
  proxy-real-ip-cidr: 10.4.0.0/16
  use-forwarded-headers: 'true'
  use-proxy-protocol: 'true'
  compute-full-forwarded-for: 'true'

To conclude our comment discussion with @night-gold, to make NGINX to accept proxy protocol you have to specifically mention that in listen directive:

http {
    #...
    server {
        listen 80   proxy_protocol;
        listen 443  ssl proxy_protocol;
        #...
    }
}

Check out NGINX guide for more.