Stateful microservices default 443 port share issue on the azure service fabric cluster

We have Cluster where multiple microservices deployed, details as follows: Total 7 microservices deployed to the Cluster, out of which 3 are stateless micro-services and 4 are stateful micro-services. Implemented http.sys for exposing secured endpoints, and want to expose all these services endpoints with default port 443. To differentiate between the services added alias names in the URL's.

Proper Load balancing rule and probe enabled to access with 443 port, and accessing these services with FQDN url's.

Stateless micro-services works fine as expected.

But unable to access stateful microservices with FQDN url's. Giving error as, HTTP Error 503. The service is unavailable.

If used specific ports for each service, its working fine, but we need to access with shared 443 port only.

Thanks in advance for suggestions.

You didn't let clear how you are exposing these services, I will assume you are accessing them directly from the load balancer to the node port 443 that the services open (and share), instead of using the reverse proxy approach described here.

Stateful Services have different behaviour that you need to understand properly while registering these ports:

Stateful services might host multiple partitions on same host(process), for that reason each replica partition might want to use the same port as each other. In this case, the correct approach as described in the docs is register a prefix containing the partition and replica id in it, if you followed the docs, you have probably registered the stateful services like this:

private ICommunicationListener CreateInternalListener(ServiceContext context)
{
 EndpointResourceDescription internalEndpoint = context.CodePackageActivationContext.GetEndpoint("ProcessingServiceEndpoint");
 string uriPrefix = String.Format(
        "{0}://+:{1}/{2}/{3}-{4}/",
        internalEndpoint.Protocol,
        internalEndpoint.Port,
        context.PartitionId,
        context.ReplicaOrInstanceId,
        Guid.NewGuid());

 string nodeIP = FabricRuntime.GetNodeContext().IPAddressOrFQDN;

 string uriPublished = uriPrefix.Replace("+", nodeIP);
 return new HttpCommunicationListener(uriPrefix, uriPublished, this.ProcessInternalRequest);
}

That will make the service acessible by an url like this:

{scheme}://{nodeIp}:{port}/{partitionid}/{replicaid}-{guid}

Another issue is,

• Stateful services might not be available on all nodes behind the load balancer;
• If they do, and they are partitioned, each partition won't be available on all nodes;
• if the partitions are on all nodes, you also have to enable secondary replica read, otherwise the endpoint won't be open.

For these reasons, you should avoid directly exposing stateful services and put it behind a proxy, like the one described in the first link.