Path based rules giving 404 for AKS ingress controller

I have two ClusterIP services on AKS and an ingress controller ingress-nginx-4.6.1 when I try the following config to route the traffic to each service I get 404.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: spark-ingress
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    nginx.ingress.kubernetes.io/rewrite-target: /$2
    nginx.ingress.kubernetes.io/use-regex: "true"
spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - spark.company.net
    secretName: tls-secret
  rules:
  - host: spark.company.net
    http:
      paths:
      - path: /service1(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: service-1
            port:
              number: 443
      - path: /service2(/|$)(.*)
        pathType: Prefix
        backend:
          service:
            name: service-2
            port:
              number: 443

the output of kubectl describe ing is:

Name:             spark-ingress
Labels:           <none>
Namespace:        apache-spark
Address:          10.0.0.0
Default backend:  default-http-backend:80 (<error: endpoints "default-http-backend" not found>)
TLS:
  tls-secret terminates spark.company.net
Rules:
  Host                  Path  Backends
  ----                  ----  --------
  spark.company.net
                        /service1(/|$)(.*) service-1:443 (100.64.101.8:4440)
                        /service1(/|$)(.*) service-2:443 (100.64.101.0:4440)
Annotations:            
                        nginx.ingress.kubernetes.io/backend-protocol: HTTPS
                        nginx.ingress.kubernetes.io/rewrite-target: /$2
                        nginx.ingress.kubernetes.io/use-regex: true
Events:
  Type    Reason  Age                    From                      Message
  ----    ------  ----                   ----                      -------
  Normal  Sync    2m9s (x3165 over 26h)  nginx-ingress-controller  Scheduled for sync
  Normal  Sync    2m9s (x3165 over 26h)  nginx-ingress-controller  Scheduled for sync

It's worth noting that the backend application is open source spark. so the default behavior when hitting the app is to 302 redirect to /jobs/. I would also like to point out that when I define the ing for one service (does not matter which) on the path / it works just fine. I need to do this for multiple services. Any help would be appreciated.

The issue was that my backend application (Apache Spark) was sending redirects to the client. so the client is now requesting http://baseurl/resource instead of http://baseurl/service1/resource

The fix was to add the following properties to the spark-submit

 --conf spark.ui.proxyBase=/service1\
 --conf spark.ui.proxyRedirectUri=localhost/service1\

For all redirects spark ui will now use service1 as base which inginx has a rule for and knows how to handle.