I am creating ingress-nginx controller using the Helm chart in Terraform. I do have a values.yaml file where I can add the customized information, but I need to pass the SSL Certificate value from the Terraform resource so how can I do that? I am using the below code but getting an error.
resource "aws_acm_certificate" "ui_cert" {
domain_name = var.DOMAIN_NAME
validation_method = "DNS"
tags = {
Environment = var.ENVIRONMENT
}
lifecycle {
create_before_destroy = true
}
}
resource "helm_release" "nginix_ingress" {
depends_on = [module.eks, kubernetes_namespace.nginix_ingress,aws_acm_certificate.ui_cert]
name = "ingress-nginx"
repository = "https://kubernetes.github.io/ingress-nginx"
chart = "ingress-nginx"
namespace = var.NGINX_INGRESS_NAMESPACE
values = [templatefile("values.yaml", {
controller.service.beta.kubernetes.io/aws-load-balancer-internal = aws_acm_certificate.ui_cert.name,
})]
}
I am getting the following error:
Error: Reference to undeclared resource
│
│ on ui.tf line 27, in resource "helm_release" "nginix_ingress":
│ 27: controller.service.beta.kubernetes.io/aws-load-balancer-internal = aws_acm_certificate.ui_cert.name,
│
│ A managed resource "controller" "service" has not been declared in the root
│ module.
╵
╷
│ Error: Invalid reference
│
│ on ui.tf line 27, in resource "helm_release" "nginix_ingress":
│ 27: controller.service.beta.kubernetes.io/aws-load-balancer-internal = aws_acm_certificate.ui_cert.name,
│
│ A reference to a resource type must be followed by at least one attribute
│ access, specifying the resource name.
╵
╷
│ Error: Unsupported attribute
│
│ on ui.tf line 27, in resource "helm_release" "nginix_ingress":
│ 27: controller.service.beta.kubernetes.io/aws-load-balancer-internal = aws_acm_certificate.ui_cert.name,
│
│ This object has no argument, nested block, or exported attribute named
│ "name".
╵
╷
│ Error: Reference to undeclared resource
│
│ on ui.tf line 27, in resource "helm_release" "nginix_ingress":
│ 27: controller.service.beta.kubernetes.io/aws-load-balancer-internal = aws_acm_certificate.ui_cert.name,
│
│ A managed resource "controller" "service" has not been declared in the root
│ module.
╵
╷
│ Error: Invalid reference
│
│ on ui.tf line 27, in resource "helm_release" "nginix_ingress":
│ 27: controller.service.beta.kubernetes.io/aws-load-balancer-internal = aws_acm_certificate.ui_cert.name,
│
│ A reference to a resource type must be followed by at least one attribute
│ access, specifying the resource name.
╵
╷
│ Error: Unsupported attribute
│
│ on ui.tf line 27, in resource "helm_release" "nginix_ingress":
│ 27: controller.service.beta.kubernetes.io/aws-load-balancer-internal = aws_acm_certificate.ui_cert.name,
│
│ This object has no argument, nested block, or exported attribute named
│ "name".
controller:
name: controller
image:
chroot: false
registry: registry.k8s.io
image: ingress-nginx/controller
tag: "v1.3.0"
digest: sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5
digestChroot: sha256:0fcb91216a22aae43b374fc2e6a03b8afe9e8c78cbf07a09d75636dc4ea3c191
pullPolicy: IfNotPresent
runAsUser: 101
allowPrivilegeEscalation: true
containerName: controller
containerPort:
# http: 80
https: 443
config:
use-proxy-protocol: "true"
# -- Optionally customize the pod hostname.
hostname: {}
# -- Process IngressClass per name (additionally as per spec.controller).
ingressClassByName: false
# -- This configuration defines if Ingress Controller should allow users to set
# their own *-snippet annotations, otherwise this is forbidden / dropped
# when users add those annotations.
# Global snippets in ConfigMap are still respected
allowSnippetAnnotations: true
## This section refers to the creation of the IngressClass resource
## IngressClass resources are supported since k8s >= 1.18 and required since k8s >= 1.19
ingressClassResource:
# -- Name of the ingressClass
name: nginx
# -- Is this ingressClass enabled or not
enabled: true
# -- Is this the default ingressClass for the cluster
default: false
# -- Controller-value of the controller that is processing this ingressClass
controllerValue: "k8s.io/ingress-nginx"
# -- Parameters is a link to a custom resource containing additional
# configuration for the controller. This is optional if the controller
# does not require extra parameters.
parameters: {}
# -- For backwards compatibility with ingress.class annotation, use ingressClass.
# Algorithm is as follows, first ingressClassName is considered, if not present, controller looks for ingress.class annotation
ingressClass: nginx
# -- Labels to add to the pod container metadata
podLabels: {}
# key: value
# -- Security Context policies for controller pods
# -- Allows customization of the source of the IP address or FQDN to report
# in the ingress status field. By default, it reads the information provided
# by the service. If disable, the status field reports the IP address of the
# node or nodes where an ingress controller pod is running.
publishService:
# -- Enable 'publishService' or not
enabled: true
# -- Allows overriding of the publish service to bind to
# Must be <namespace>/<service_name>
pathOverride: ""
tcp:
# -- Allows customization of the tcp-services-configmap; defaults to $(POD_NAMESPACE)
configMapNamespace: ""
# -- Annotations to be added to the tcp config configmap
annotations: {}
udp:
# -- Allows customization of the udp-services-configmap; defaults to $(POD_NAMESPACE)
configMapNamespace: ""
# -- Annotations to be added to the udp config configmap
annotations: {}
# -- Use a `DaemonSet` or `Deployment`
kind: Deployment
# -- Annotations to be added to the controller Deployment or DaemonSet
##
annotations: {}
# keel.sh/pollSchedule: "@every 60m"
# -- Labels to be added to the controller Deployment or DaemonSet and other resources that do not have option to specify labels
##
labels: {}
# keel.sh/policy: patch
# keel.sh/trigger: poll
# -- The update strategy to apply to the Deployment or DaemonSet
##
updateStrategy: {}
# rollingUpdate:
# maxUnavailable: 1
# type: RollingUpdate
# -- `minReadySeconds` to avoid killing pods before we are ready
##
minReadySeconds: 0
# -- Topology spread constraints rely on node labels to identify the topology domain(s) that each Node is in.
## Ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-topology-spread-constraints/
##
topologySpreadConstraints: []
# - maxSkew: 1
# topologyKey: topology.kubernetes.io/zone
# whenUnsatisfiable: DoNotSchedule
# labelSelector:
# matchLabels:
# app.kubernetes.io/instance: ingress-nginx-internal
# -- `terminationGracePeriodSeconds` to avoid killing pods before we are ready
## wait up to five minutes for the drain of connections
##
terminationGracePeriodSeconds: 300
# -- Node labels for controller pod assignment
## Ref: https://kubernetes.io/docs/user-guide/node-selection/
##
nodeSelector:
kubernetes.io/os: linux
## Liveness and readiness probe values
## Ref: https://kubernetes.io/docs/concepts/workloads/pods/pod-lifecycle/#container-probes
##
## startupProbe:
## httpGet:
## # should match container.healthCheckPath
## path: "/healthz"
## port: 10254
## scheme: HTTP
## initialDelaySeconds: 5
## periodSeconds: 5
## timeoutSeconds: 2
## successThreshold: 1
## failureThreshold: 5
livenessProbe:
httpGet:
# should match container.healthCheckPath
path: "/healthz"
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 5
readinessProbe:
httpGet:
# should match container.healthCheckPath
path: "/healthz"
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
timeoutSeconds: 1
successThreshold: 1
failureThreshold: 3
# -- Path of the health check endpoint. All requests received on the port defined by
# the healthz-port parameter are forwarded internally to this path.
healthCheckPath: "/healthz"
# -- Address to bind the health check endpoint.
# It is better to set this option to the internal node address
# if the ingress nginx controller is running in the `hostNetwork: true` mode.
healthCheckHost: ""
# -- Annotations to be added to controller pods
##
podAnnotations: {}
replicaCount: 1
minAvailable: 1
## Define requests resources to avoid probe issues due to CPU utilization in busy nodes
## ref: https://github.com/kubernetes/ingress-nginx/issues/4735#issuecomment-551204903
## Ideally, there should be no limits.
## https://engineering.indeedblog.com/blog/2019/12/cpu-throttling-regression-fix/
resources:
## limits:
## cpu: 100m
## memory: 90Mi
requests:
cpu: 100m
memory: 90Mi
# Mutually exclusive with keda autoscaling
autoscaling:
enabled: false
minReplicas: 1
maxReplicas: 11
targetCPUUtilizationPercentage: 50
targetMemoryUtilizationPercentage: 50
behavior: {}
# scaleDown:
# stabilizationWindowSeconds: 300
# policies:
# - type: Pods
# value: 1
# periodSeconds: 180
# scaleUp:
# stabilizationWindowSeconds: 300
# policies:
# - type: Pods
# value: 2
# periodSeconds: 60
autoscalingTemplate: []
# Custom or additional autoscaling metrics
# ref: https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-custom-metrics
# - type: Pods
# pods:
# metric:
# name: nginx_ingress_controller_nginx_process_requests_total
# target:
# type: AverageValue
# averageValue: 10000m
# Mutually exclusive with hpa autoscaling
keda:
apiVersion: "keda.sh/v1alpha1"
## apiVersion changes with keda 1.x vs 2.x
## 2.x = keda.sh/v1alpha1
## 1.x = keda.k8s.io/v1alpha1
enabled: false
minReplicas: 1
maxReplicas: 11
pollingInterval: 30
cooldownPeriod: 300
restoreToOriginalReplicaCount: false
scaledObject:
annotations: {}
# Custom annotations for ScaledObject resource
# annotations:
# key: value
triggers: []
# - type: prometheus
# metadata:
# serverAddress: http://<prometheus-host>:9090
# metricName: http_requests_total
# threshold: '100'
# query: sum(rate(http_requests_total{deployment="my-deployment"}[2m]))
behavior: {}
# scaleDown:
# stabilizationWindowSeconds: 300
# policies:
# - type: Pods
# value: 1
# periodSeconds: 180
# scaleUp:
# stabilizationWindowSeconds: 300
# policies:
# - type: Pods
# value: 2
# periodSeconds: 60
# -- Enable mimalloc as a drop-in replacement for malloc.
## ref: https://github.com/microsoft/mimalloc
##
enableMimalloc: true
## Override NGINX template
customTemplate:
configMapName: ""
configMapKey: ""
service:
enabled: true
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "ssl-cert"
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
Can someone plz tell me how I can make it work?
The templatefile built-in function is used to pass the values to variables defined in the templated file. So for example, in your case, you would define a variable in the template file called e.g., ssl_cert. Then, when calling the templatefile function, you would pass it the value provided by the ACM resource:
values = [templatefile("values.yaml", {
ssl_cert = aws_acm_certificate.ui_cert.name
})]
The ssl_cert variable inside of the templatefile would be associated with the annotation controller.service.beta.kubernetes.io/aws-load-balancer-ssl-cert. Based on the YML file, the variable should be added in the last section of the file:
service:
enabled: true
annotations:
service.beta.kubernetes.io/aws-load-balancer-type: "nlb"
service.beta.kubernetes.io/aws-load-balancer-proxy-protocol: "*"
service.beta.kubernetes.io/aws-load-balancer-ssl-cert: "${ssl_cert}" # here is where the replacement will be made
service.beta.kubernetes.io/aws-load-balancer-backend-protocol: "http"
service.beta.kubernetes.io/aws-load-balancer-ssl-ports: "443"
The templatefile function is really powerful, but I strongly suggest understanding how variables and variable substitution works when using it [1]. It is unaware of the substitution you want to make unless you have a placeholder variable both when calling the function and in the template file.
[1] https://www.terraform.io/language/functions/templatefile