google-kubernetes-enginegoogle-compute-enginekubernetes-helmkubernetes-ingressingress-controller

Helm fails to add TLS in the GKE ingress


I'm trying to deploy the application in the GKE which contains ingress details

values.yaml

# Default values for test-frontend.
# This is a YAML-formatted file.
# Declare variables to be passed into your templates.

global:
  namespace: isp

replicaCount: 2

image:
  repository: test.code.com/test/test-frontend
  pullPolicy: Always
  # Overrides the image tag whose default is the chart appVersion.
  tag: ""

imagePullSecrets:
  - name: docker-cfg        
nameOverride: ""
fullnameOverride: "test-frontend"

serviceAccount:
  # Specifies whether a service account should be created
  create: false
  # Annotations to add to the service account
  annotations: {}
  # The name of the service account to use.
  # If not set and create is true, a name is generated using the fullname template
  name: "cloudsql-sa"

podAnnotations: {}

podSecurityContext: {}
  # fsGroup: 2000

securityContext: {}
  # capabilities:
  #   drop:
  #   - ALL
  # readOnlyRootFilesystem: true
  # runAsNonRoot: true
  # runAsUser: 1000

service:
  name: isp-stabsuite-frontend-service
  annotations:
    cloud.google.com/neg: '{"ingress": true}'
  type: NodePort
  port: 80

resources:
  limits:
    cpu: 350m
    memory: 1000Mi
  requests:
    cpu: 250m
    memory: 750Mi

  # We usually recommend not to specify default resources and to leave this as a conscious
  # choice for the user. This also increases chances charts run on environments with little
  # resources, such as Minikube. If you do want to specify resources, uncomment the following
  # lines, adjust them as necessary, and remove the curly braces after 'resources:'.
  # limits:
  #   cpu: 350m
  #   memory: 1000Mi
  # requests:
  #   cpu: 250m
  #   memory: 750Mi

autoscaling:
  enabled: false
  minReplicas: 1
  maxReplicas: 100
  targetCPUUtilizationPercentage: 80
  # targetMemoryUtilizationPercentage: 80

nodeSelector: {}

tolerations: []

affinity: {}

#########Required Modifications###############

configmap:
  name: test-frontend-configmap
  env:
    GET_HOSTS_FROM: "dns"
    REACT_APP_BACKEND_API: "/api/v1"
    REACT_APP_GOOGLE_CLIENT_ID: ""
    APP_DB_CONNECTION_STRING: ""
    NODE_ENV: "production"
ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: "gce-internal"
    kubernetes.io/ingress.regional-static-ip-name : "ingress-internal-static-ip"
    kubernetes.io/ingress.allow-http: "false"
  hosts:
    - host: test-sandbox.test.com
      paths:
      - path: "/"
        servicename: test-frontend-service
        serviceport: 80
      - path: "/api/"
        servicename: test-backend-service
        serviceport: 80
  tls:
  - hosts:
    - test-sandbox.test.com
    secretName: ingress-tls-credential-file 
    type: kubernetes.io/tls

#########Required Subchart Modifications###
isp-stabsuite-backend:
  cloudsqlConnection: ""
  cloudsqlproxySa: ""
  service:
    name: test-backend-service

  bigquerysecret:
    name: gcp-bigquery-credential-file
    type: Opaque
    content:
      credentials: test

ingress-template file

{{- if .Values.ingress.enabled -}}
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: {{ .Values.global.namespace }}-ingress
  namespace: {{ .Values.global.namespace }}
  labels:
    {{- include "test-frontend.labels" . | nindent 4 }}
  annotations:
    kubernetes.io/ingress.class: "gce-internal"
    kubernetes.io/ingress.allow-http: "false"
    kubernetes.io/ingress.global-static-ip-name : "ingress-internal-static-ip" 
    cloud.google.com/neg: '{"ingress": true}'
spec:
  {{- if .Values.ingress.tls }}
  tls:
     {{- range .Values.ingress.tls }}
     - hosts:
         {{- range .hosts }}
         - {{ . | quote }}
         {{- end }}
       secretName: {{ .secretName }}
     {{- end }}
  {{- end }} 
  rules:
    {{- range .Values.ingress.hosts }}
    - host: {{ .host | quote }}
      http:
        paths:
          {{- range .paths }}
          - path: {{ .path }}
            pathType: Prefix
            backend:
              service:
                name: {{ .servicename }}
                port:
                  number: {{ .serviceport }}
          {{- end }}
    {{- end }}
  {{- end }}

When it tries to create ingress, it thows below error

Error syncing to GCP: error running load balancer syncing routine: loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid configuration: both HTTP and HTTPS are disabled (kubernetes.io/ingress.allow-http is false and there is no valid TLS configuration); your Ingress will not be able to serve any traffic

Even I cleanup the ingress and try to install fresh ingress it throws same error with same LoadBalancer ID (loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i)

kubectl describe ingress test-ingress -n test
Name:             test-ingress
Labels:           app.kubernetes.io/managed-by=Helm
                  app.kubernetes.io/name=test-frontend
                  app.kubernetes.io/type=frontend
                  app.kubernetes.io/version=1.16.0
                  helm/chart=test-frontend-0.1.0
Namespace:        isp
Address:
Ingress Class:    <none>
Default backend:  <default>
Rules:
  Host                         Path  Backends
  ----                         ----  --------
  test-sandbox.test.com
                               /       test-frontend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
                               /api/   test-backend-service:80 (xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80,xxx.xx.xxx.xx:80)
Annotations:                   cloud.google.com/neg: {"ingress": true}
                               kubernetes.io/ingress.allow-http: false
                               kubernetes.io/ingress.class: gce-internal
                               kubernetes.io/ingress.regional-static-ip-name: ingress-internal-static-ip
                               meta.helm.sh/release-name: test-frontend
                               meta.helm.sh/release-namespace: default
Events:
  Type     Reason  Age                 From                     Message
  ----     ------  ----                ----                     -------
  Normal   Sync    113s (x4 over 21m)  loadbalancer-controller  Scheduled for sync
  Warning  Sync    24s (x19 over 18m)  loadbalancer-controller  Error syncing to GCP: error running load balancer syncing routine: loadbalancer z30rnomw-isp-isp-ingress-pjgwvt2i does not exist: invalid configuration: both HTTP and HTTPS are disabled (kubernetes.io/ingress.allow-http is false and there is no valid TLS configuration); your Ingress will not be able to serve any traffic

Helm Chart fails to add the TLS details in the ingress.yaml. Do I miss something?


Solution

  • From the looks of it, your values.yaml is wrong. The tls block starts at the wrong indentation. It should be at the same level as the enabled, annotations and hosts i.e. at the first level after ingress.