azurenginxazure-devopsterraform

Azure Container App via Terraform - platform_reserved_dns_ip_address is empty, not able to dynamically use NGINX as reverse proxy


I'm deploying nginx as a reverse proxy in an Azure Container App environment.

As I'm proxy-passing dynamically, I need to explicitly set a resolver/dns ip address in the NGINX config.

I was on consumption plan until now, and had the resolver hardcoded to 10.0.0.2, which worked fine.

I now switched to a dedicated workload profile, with which that address no longer worked. It now seems to be changing dynamically upon environment creation.

Thus, I tried to dynamically alter the config file via terraform by using the app environment's platform_reserved_dns_ip_address. However, this value is just empty and does not pass a validation like below.

As per terraform documentation, infrastructure_subnet_id is configured so that should not be the issue.

Is there any way to either retrieve that resolver ip or workaround differently to be able to dynamically proxy in an ACA environment using NGINX?

Some code snippets:

# terraform excerpts from different modules

resource "azurerm_container_app_environment" "main" {
  name                           = "foo"
  location                       = var.location
  resource_group_name            = var.resource_group_name
  log_analytics_workspace_id     = azurerm_log_analytics_workspace.main.id
  infrastructure_subnet_id       = azurerm_subnet.containerEnv.id
  internal_load_balancer_enabled = var.ingress_vnet_only

  # note I'm only having the issue in dedicated, ie not consumption plan
  workload_profile {
    name                  = local.workload_profile.name
    workload_profile_type = local.workload_profile.workload_profile_type
    minimum_count         = local.workload_profile.minimum_count
    maximum_count         = local.workload_profile.maximum_count
  }
}

# output app environment
output "dns_ip_address" {
  value = azurerm_container_app_environment.main.platform_reserved_dns_ip_address
}

# input nginx
variable "dns_ip_address" {
  description = "IP address of the DNS server"
  type        = string

  validation {
    condition     = length(var.dns_ip_address) > 0
    error_message = "dns_ip_address must not be empty"
  }
}

# dynamic nginx config adaptation
resource "local_file" "nginx_config" {
  content = templatefile("${path.module}/config/nginx-template.conf", {
    dns_ip_address = var.dns_ip_address
  })
  filename = "${path.module}/auto-generated/nginx.conf"
}
# nginx conf excerpt
    resolver ${dns_ip_address}; 
    server {
        listen 80;
        server_name ~^(.*)\.foo\.bar$;

        location / {
            set $upstream_url http://$dynamic_address

            proxy_pass $upstream_url;
            proxy_http_version 1.1;
        }
    }

Also the JSON view of the app environment will say the IP address is null: "platformReservedDnsIP": null


Solution

  • I have now opened an issue here and have figured out a workaround, which uses the fact that the name server will still be present in a container's /etc/resolv.conf. It will just replace a variable in the config before NGINX is started:

    # terraform excerpt
        container {
          name   = local.nginx
          image  = "${local.nginx}:${local.nginx_version}"
          cpu    = "0.5"
          memory = "1Gi"
          # As retrieving the dns ip directly via azurerm_container_app_environment.platform_reserved_dns_ip_address does not work in dedicated plan, we replace it manually.
          command = [
            "/bin/sh", "-c",
            <<-EOT
            echo "Starting entrypoint script...";
            DNS_IP=$(grep -m 1 'nameserver' /etc/resolv.conf | awk '{print $2}');
            [ -z "$DNS_IP" ] && echo 'No nameserver found' && exit 1;
            echo "Found DNS IP: $DNS_IP";
            cp /etc/nginx/nginx.conf /tmp/nginx.conf; \
            sed -i "s/PLACEHOLDER_DNS_IP/$DNS_IP/" /tmp/nginx.conf;
            echo "Nginx config:";
            cat /tmp/nginx.conf;
            exec nginx -c /tmp/nginx.conf -g 'daemon off;';
            EOT
          ]
    
          volume_mounts {
            name = "nginx-config"
            path = "/etc/nginx/"
          }
        }