argocdexternal-dnsaws-load-balancer-controller

Combining ExternalDNS + AWS Load Balancer Controller for ArgoCD

I've been following the ArgoCD docs on AWS Application Load Balancers (ALBs) And Classic ELB (HTTP Mode) but when I mash that config together ExternalDNS seems to want use internal IPs instead of external ones:

$ dig argocd.<MyURL>.com

; <<>> DiG 9.10.6 <<>> argocd.<MyURL>.com
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3362
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 512
;; QUESTION SECTION:
;argocd.<MyURL>.com. IN      A

;; ANSWER SECTION:
argocd.<MyURL>.com. 300 IN   A       10.0.3.59
argocd.<MyURL>.com. 300 IN   A       10.0.2.77
argocd.<MyURL>.com. 300 IN   A       10.0.2.166
argocd.<MyURL>.com. 300 IN   A       10.0.1.194

;; Query time: 22 msec
;; SERVER: 8.8.8.8#53(8.8.8.8)
;; WHEN: Wed Jul 26 17:37:38 PDT 2023
;; MSG SIZE  rcvd: 122

Here's the relevant config of the 2 services and the ingress for ArgoCD:

apiVersion: v1
kind: Service
metadata:
  annotations:
    external-dns.alpha.kubernetes.io/hostname: argocd.<MyURL>.com
    meta.helm.sh/release-name: argo-cd
    meta.helm.sh/release-namespace: argo-cd
  creationTimestamp: "2023-07-14T18:26:17Z"
  labels:
    app.kubernetes.io/component: server
    app.kubernetes.io/instance: argo-cd
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: argocd-server
    app.kubernetes.io/part-of: argocd
    app.kubernetes.io/version: v2.7.6
    helm.sh/chart: argo-cd-5.36.11
  name: argo-cd-argocd-server
  namespace: argo-cd
spec:
  ports:
  - name: http
    nodePort: 30080
    port: 80
    protocol: TCP
    targetPort: 8080
  - name: https
    nodePort: 30443
    port: 443
    protocol: TCP
    targetPort: 8080
  selector:
    app.kubernetes.io/instance: argo-cd
    app.kubernetes.io/name: argocd-server
  sessionAffinity: None
  type: NodePort
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    #This tells AWS to send traffic from the ALB using HTTP2. Can use GRPC as well if you want to leverage GRPC specific features
    alb.ingress.kubernetes.io/backend-protocol-version: HTTP2 
  labels:
    app: argogrpc
  name: argogrpc
spec:
  ports:
  - name: "443"
    port: 443
    protocol: TCP
    targetPort: 8080
  selector:
    app.kubernetes.io/name: argocd-server
  sessionAffinity: None
  type: NodePort
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: argocd
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/backend-protocol: HTTPS
    # Use this annotation (which must match a service name) to route traffic to HTTP2 backends.
    alb.ingress.kubernetes.io/conditions.argogrpc: |
      [{"field":"http-header","httpHeaderConfig":{"httpHeaderName": "Content-Type", "values":["application/grpc"]}}]
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}]'
spec:
  rules:
  - host: argocd.<MyURL>.com
    http:
      paths:
      - path: /
        backend:
          service:
            name: argo-cd-argocd-server
            port:
              name: https
        pathType: Prefix
      - path: /
        backend:
          service:
            name: argogrpc
            port:
              number: 443
        pathType: Prefix

How would I go about configuring ExternalDNS and AWS Load Balancer Controller for ArgoCD?

FWIW: I do have ExternalDNS and AWS Load Balancer Controller working with my EKS Kubernetes cluster so that with a few annotations I can create ALBs and have them routed to my cluster with external DNS entries. It's just ArgoCD giving me trouble. Here's an example of a working config for an app that just serves plain HTML over port 80:

apiVersion: v1
kind: Service
metadata:
  name: feature-flag-tests
  annotations:
    external-dns.alpha.kubernetes.io/hostname: fftests.<MyURL>.com
spec:
  type: NodePort
  ports:
    - port: 80
      name: web
      protocol: TCP
      targetPort: web
  selector:
    app: feature-flag-tests
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: feature-flag-tests
  annotations:
    alb.ingress.kubernetes.io/scheme: internet-facing
    alb.ingress.kubernetes.io/target-type: ip
    alb.ingress.kubernetes.io/listen-ports: '[{"HTTPS":443}, {"HTTP":80}]'
    alb.ingress.kubernetes.io/ssl-redirect: '443'
    alb.ingress.kubernetes.io/certificate-arn: <My Cert ARN>
spec:
  ingressClassName: alb
  rules:
    - host: fftests.<myurl>.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: feature-flag-tests
                port:
                  number: 80
Solution

  • I was able to get things working by setting the ArgoCD server to run with the insecure flag (aka serve HTTP rather than HTTPS).

    I didn't create a new service, just added the external-dns.alpha.kubernetes.io/hostname to the one created by the Helm chart. I did create a new Ingress with type ingress (basically the same as the one shown in my question). I don't think you can route HTTP2 and gRPC traffic through this setup but don't know for sure (I just wanted to access the web UI).

    I still would be curious if there's a good way to make this work without the --insecure flag (and maybe with HTTP2/gRPC support) so I'd be happy to hear about other approaches to do that.