consulenvoyproxymesh-network

Consul & Envoy Integration

Background

I came from HAproxy background and recently there is a lot of hype around "Service Mesh" Architecture. Long story short, I began to learn "Envoy" and "Consul".

I develop an understanding that Envoy is a proxy software but using sidecar to abstract in-out network with "xDS" as Data Plane for the source of truth (Cluster, Route, Filter, etc). Consul is Service Discovery, Segmentation, etc. It also abstracts network and has Data Plane but Consul can't do complex Load Balancing, filter routing as Envoy does.

As Standalone, I can understand how they work and set up them since documentation relatively good. But it can quickly became a headache if I want to integrate Envoy and Consul, since documentation for both Envoy & Consul lacks specific for integration, use-cases, and best practice.

Schematic

Consider the following simple infrastructure design:

enter image description here

Legends:

Questions

Following are my questions:

  1. In the event of Multi-Instance Microservices, Consul (as stand-alone) will randomize round-robin. With Envoy & Consul Integration, How consul handle multi-instance microservice? which software does the load balance?
  2. Consul has Consul Server to store its data, however, it seems Envoy does not have "Envoy Server" to store its data, so where are its data being stored and distributed across multiple instances?
  3. What about Envoy Cluster (Logical Group of Envoy Front Facing Proxy and NOT Cluster of Services)? How the leader elected?
  4. As I mentioned above, Separately, Consul and Envoy have their sidecar/agent on each Machine. I read that when integrated, Consul injects Envoy Sidecar, but no further information on how this works?
  5. If Envoy uses Consul Server as "xDS", what if for example I want to add an advanced filter so that for certain URL segment it must forward to a certain instance?
  6. If Envoy uses Consul Server as "xDS", what if I have another machine and services (for some reason) not managed by Consul Server. How I configure Envoy to add filter, cluster, etc for that machine and services?

Thank You, I'm so excited I hope this thread can be helpful to others too.

Solution

  • Apologies for the late reply. I figure its better late than never. :-)

    1. If you are only using Consul for service discovery, and directly querying it via DNS then Consul will randomize the IP addresses returned to the client. If you're querying the HTTP interface, it is up to the client to implement a load balancing strategy based on the hosts returned in the response. When you're using Consul service mesh, the load balancing function will be entirely handled by Envoy.
    2. Consul is an xDS server. The data is stored within Consul and distributed to the agents within the cluster. See the Connect Architecture docs for more information.
    3. Envoy clusters are similar to backend server pools. Proxies contain Clusters for each upstream service. Within each cluster, there are Endpoints which represent the individual proxy instances for the upstream services.
    4. Consul can inject the Envoy sidecar when it is deployed on Kubernetes. It does this through a Kubernetes mutating admission webhook. See Connect Sidecar on Kubernetes: Installation and Configuration for more information.
    5. Consul supports advanced layer 7 routing features. You can configure a service-router to route requests to different destinations by URL paths, headers, query params, etc.
    6. Consul has an upcoming feature in version 1.8 called Terminating Gateways which may enable this use case. See the GitHub issue "Connect: Terminating (External Service) Gateways" (hashicorp/consul#6357) for more information.