google-cloud-platformjmetertraefiktraefik-ingressgcp-load-balancer

Troubleshooting high latency spikes on Google Global HTTPS Load Balancer with Traefik


I'm experiencing significant latency spikes when fetching a 2.5MB JAR file (/client/signed-jars/lib/bc-prov.jar) using JMeter against a Google Global HTTPS Load Balancer. Out of 250 identical GET requests, 84 take over 500ms, with some reaching 10 seconds.

Goal: I want to identify the specific requests experiencing high latency within the load balancer and Traefik. Ideally, I'd like to track how long each request takes to process at each component.

Question: Is there a way to uniquely identify each of the 250 JMeter requests, perhaps by adding a counter or a unique identifier? This would allow me to correlate the requests with the corresponding log entries in the load balancer and Traefik.

Setup:

Client: JMeter Load Balancer: Google Global HTTPS Load Balancer Ingress Controller: Traefik Backend: VM hosting the application Problem: When directly accessing the VM via its public IP, the average latency for 250 requests is a mere 2ms. This indicates that the issue lies within the load balancer and/or Traefik.


Solution

  • Normally people use HTTP Headers for this, for example in APM systems such approach is widely used for tracing the request across multiple applications and services.

    Header name can be whatever you want, i.e. X-Request-Id, and the value - you can either go for an unique identifier from __UUID() function or it can be current thread (virtual user) number from __threadNum() function and iteration number from pre-defined variables.

    A header can be added to the request via HTTP Header Manager