I've been hemorrhaging hours of research to this and cannot seem to find the right solution, even to understand the problem itself.
The scenario: I have an multiple ubuntu servers, each server runs an apache hosting multiple sites via apache-vhost. I have a CloudFlare free plan and configures it to only do Flexible SSL/TSL and a Page Rule to force HTTPS access.
The problem: Some client gets Cloudflare 524 Error while some are just fine.
Tests/Observations:
If I disabled the Page Rule force HTTPS, everything is fine. So this should point on the https problem, but why does the https works on some, and not on the other clients?
http requests are properly logged in the access-log of each vhosted site, I tried it myself, it actually logged my request but I still get Error 524.
The servers are not overloaded but I still get Error 524.
Increased the mpm_prefork settings, still Error 524.
Disallowed any IPs except cloudflare ips via iptables, still Error 524
Disabled multiple vhosted site, still Error 524
I'm currently out of idea on how to understand this problem to solve it. I've been exhausing weeks of research but all threads mentioning this specific error doesn't fit with my case. Help!
After 3 years, I think I found the culprit. This is just my hunch but it fixed the problem for me.
My servers has two network interfaces, one for LAN and one for WAN (which I use for connecting to cloudflare). The problem is, I didn't set a proper route metric value (prioritization) on which interface the server should prioritize to use.
I think, just a hunch, because of this, the server is trying to resolve which connection to use when sending back the response that's why its a hit or miss. I don't know why its an issue when on HTTPS but setting the route metric value for me fixed the problem and made my sites responds noticibly faster.
To know more about route-metric: https://netplan.readthedocs.io/en/0.106/examples/#connecting-multiple-interfaces-with-dhcp