kubernetesazure-aksservicemesh

AKS pod with two services within OSM


We have an application which exposes two ports (for API and WebSocket). Application is deployed in OSM-enabled namespace. We're using nginx-ingress for external access. Currently there are:

#api-svc
Type:              ClusterIP
IP:                [some-ip]
Port:              http  80/TCP
TargetPort:        18610/TCP
Endpoints:         [some-ip]:18610
-------
#websocket-svc
Type:              ClusterIP
IP:                [some-ip]
Port:              ws  80/TCP
TargetPort:        18622/TCP
Endpoints:         [some-ip]:18622
paths:
 - path: /api
   pathType: ImplementationSpecific
   backend:
    service:
     name: api-svc
     port:
      number: 80
 - path: /swiftsockjs
   pathType: ImplementationSpecific
   backend:
    service:
     name: websocket-svc
     port:
       number: 80
Spec:
  Backends:
    Name:  api-svc
    Port:
      Number:    18610
      Protocol:  http
    Name:        websocket-svc
    Port:
      Number:    18622
      Protocol:  http
  Sources:
    Kind:       Service
    Name:       ingress-nginx-controller
    Namespace:  ingress

The problem we are facing is that traffic is routed only to one targetPort at a time (i.e. only to 18610 or 18622) regardless the URL path. In the ingress controller logs it’s visible to traffic is routed correctly (/api to 18610 and /swiftsockjs to 18622). The problem is visible in the envoy sidecar logs. Both requests are going to the same upstream_cluster (it should differ by port). This can be seen at line 15th of comparision: Comparision

What's the strangest the behavior is changing randomly when service or ingressBackend are redeployed. So one time all requests are forwarded to 18610 and other time to 18622.

We have tried to use multi-port service but according to this OSM PR it's not supported (although results were exactly the same).

Does anyone has any ideas how to fix this? I've read almost whole OSM documentation and MS Docs regarding OMS-Addon but haven't find answer to this problem (or similar example with multiport pod in OSM).


Solution

  • According to Azure support - such a solution is not possible within OSM. Quote:

    A restart of the process or the pod sometimes results in the IP:PORT change but also traffic will be consistently forwarded to that IP:PORT.

    This appears to be due to the behavior of the proxy. As per OSM github document. It is a 1:1 relationship between the proxy and the endpoint. It is also a 1:1 relationship between the proxy and the service.

    In other words, the proxy will not be able to handle a pod serving multiple services.

    Suggestion from MS was to split application logic to separate deployments(pods) so both can server one port at a time.