I'm integrating Cilium with Istio in a Kubernetes environment and need clarity on how specific configurations will impact this integration. Based on Cilium's official doc, the two key configurations in question are:
Configuration A: Running Cilium with kube-proxy enabled.
Configuration B: Running Cilium without kube-proxy, using kubeProxyReplacement.
The goal is to understand the specific implications of each configuration on Istio's functionality and stability. My primary concern is ensuring that traffic to Istio’s sidecar proxies is not disrupted, especially in light of Cilium’s socket-based load balancing (socket-lb).
How would each configuration interact with Istio's sidecar proxies?
Are there known issues or significant differences in performance, network stability, or compatibility with Istio between these two configurations?
Insights, particularly from those who have experience with Cilium and Istio in production environments, would be invaluable.
The socket-level load balancing component of Cilium's kube-proxy replacement would indeed cause the sidecar redirection to be skipped.
To avoid that, Cilium includes an option to disable socket-level load balancing for managed pods. It can be enabled with socketLB.hostNamespaceOnly=true (Helm) or bpf-lb-sock-hostns-only=true (Cilium ConfigMap). See the related documentation for more information.
I'm not aware of any other incompatibilities between Istio and Cilium's kube-proxy replacement.