container engine ingress not working

I have a simple container in google container registry which basically does a few things and executes a binary which is a go based server, here are the contents of the DockerFile:

FROM debian:stable

WORKDIR /workspace/

COPY key.json .
COPY bin/user-creds.

EXPOSE 1108

ENV GOOGLE_APPLICATION_CREDENTIALS /workspace/key.json

RUN apt-get update \
    && apt-get install -y ca-certificates \
    && chmod +x user-creds

CMD ["./user-creds"]

this container has been tested locally and works perfectly. So using the google cloud shell I ran this container:

kubectl run user-creds --image=eu.gcr.io/GCLOUD_PROJECT/user-creds:COMMIT_SHA --port=1108

Then like it says on the doc, i exposed it on a nodeport

kubectl expose deployment user-creds --target-port=1108 --type=NodePort

Then I created an ingress with a path to the sevice:

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: INGRESS_NAME
  annotations:
      kubernetes.io/ingress.global-static-ip-name: IP_NAME
spec:
  rules:
    - http:
        paths:
        - path: /user/creds/*
          backend:
            serviceName: user-creds
            servicePort: 1108

then i created the ingress:

kubectl create -f INGRESS_NAME.yaml

the ingress was created and i waited some time, here is the details of the ingress:

NAME                     HOSTS     ADDRESS        PORTS     AGE
INGRESS_NAME             *         IP_ADDRESS     80        38m

but when i go the the actual url with the path I get a 502 error:

When I go to any other path I get the default backend 404 error but when i visit the specific /user/creds/ path i get the 502 error.

To check if it is something wrong with the cluster or my specific container, port or something else, I tried exposing the container as a LoadBalancer and it works perfectly, the Command:

kubectl expose deployment user-creds --target-port=1108 --port=80 --type=LoadBalancer

service details:

NAME         TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
kubernetes   ClusterIP      INT_IP_ADDRESS  <none>          443/TCP        1h
user-creds   LoadBalancer   INT_IP_ADDRESS  IP_ADDRESS      80:31618/TCP   1m

result: 200 with the correst response body.

Been stuck on this for time now, tried the ingress with no paths just the user-creds as the backend but still has the same error.

Any help or suggestion would be appreciated, thanks :)

Finally figured it out, it was to do with the health check. The health check visits / and expects a 200, if it doesn't get it then it marks the backend as unhealthy and returns 502 for every requests sent to it. My problem was that I was using the / endpoint which would've normally returned a 400 if its being called with no specific request parameters.

It was really a human error on my side, it even specifically said that in the docs: https://cloud.google.com/kubernetes-engine/docs/tutorials/http-balancer#remarks

Another thing to consider is that the ingress returns all the the paths before the route so the the server needs to literally listen for /user/creds/ in my case.