loadBalancerIP and annotations in nginx-ingress fluxcd HelmRelease are ignored on azure AKS

I am operating on an Azure AKS cluster, where fluxcd is working properly. I am trying to configure the nginx-ingress chart via the fluxcd CDR HelmRelease. Specifically to tell nginx-ingress to use an already created static public IP which lies within another resource group as the AKS node resource group.

The underlying AKS service principal has network contributor role on the resource group where the ip lies.

When using the following HelmRelease config file, the loadBalancerIP property (as well as any annotation) is ignored and a new public IP within azure is created an assigned to the azure load balancer.

apiVersion: helm.fluxcd.io/v1
kind: HelmRelease
metadata:
  name: helm-nginx-ingress
  namespace: nginx-ingress
spec:
  releaseName: nginx-ingress
  targetNamespace: nginx-ingress
  chart:
    repository: https://kubernetes-charts.storage.googleapis.com/
    name: nginx-ingress
    version: 1.24.4
  values:
    controller.service:
      loadBalancerIP: "12.234.162.41"
      annotations:
        service.beta.kubernetes.io/azure-load-balancer-resource-group: "name-of-resource-group"

To make things easier I also tried with an ip which directly exists in the same resource group as the AKS nodes (to no avail). BTW: There are no errors whats so ever. It just ignores the property and creates a new ip.

Interestingly enough, releasing the same chart with helm 3 directly (so without fluxcd) worked perfectly, even with the special case of the ip not being in the same resource group:

helm upgrade nginx-ingress --install --wait --namespace nginx-ingress \
--set controller.service.loadBalancerIP="12.234.162.41" \
--set controller.service.annotations."service\.beta\.kubernetes\.io/azure-load-balancer-resource-group"="name-of-resource-group" \
stable/nginx-ingress

So my guess is the problem lies somewhere with the CDR HelmRelease and the translation of the values spec. Dis someone experience similar problems?

AKS Kubernetes Version: v1.14.8
fluxcd/flux image tag: 1.17.0
Local Helm version: v3.0.3

Should be this:

values:
  controller:
    service:
      loadBalancerIP: "12.234.162.41"
      xxx

https://github.com/helm/charts/blob/master/stable/nginx-ingress/values.yaml#L246