I have an ingress gateway set up which all works perfectly and can route traffic to services through VirtualServices. I've also set up an EnvoyFilter using the jwt_authn_filter to perform some authentication on certain routes for incoming traffic to my cluster. This EnvoyFilter also adds a custom header to traffic based on one of the claims within the JWT:
apiVersion: networking.istio.io/v1alpha3
kind: EnvoyFilter
metadata:
name: jwt-auth-filter
namespace: istio-system
spec:
workloadSelector:
labels:
istio: ingress-gateway
configPatches:
- applyTo: HTTP_FILTER
match:
context: GATEWAY
listener:
portNumber: 443
filterChain:
filter:
name: envoy.filters.network.http_connection_manager
subFilter:
name: envoy.filters.http.router
patch:
operation: INSERT_FIRST
value:
name: envoy.filters.http.jwt_authn
typed_config:
"@type": type.googleapis.com/envoy.extensions.filters.http.jwt_authn.v3.JwtAuthentication
providers:
x:
...
claim_to_headers:
- header_name: x-custom
claim_name: custom
...
This all works fine, I can see the header being added in the logs and my services that are routed to receive this header. However, if I try to use this header in a Virtual Service, it doesn't work, I get a 404 NR route_not_found error.
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
name: vs1
namespace: ns1
spec:
gateways:
- istio-system/ingress-gateway
hosts:
- 'mydomain.com'
http:
- name: vs1
match:
- headers:
x-custom:
exact: value
route:
- destination:
host: s1.ns1.svc.cluster.local
port:
number: 80
It seems like the headers for routing are determined before the above filter runs. Is this correct, and if so, is there a better alternative approach?
The VirtualService itself is working fine, if I manually set that header on requests to the ingress gateway it gets routed correctly so there aren't any issues there.
Turns out there is an undocumented option clear_route_cache which solves this issue.