cloudflarecdnamazonsellercentral

I don't understand the cache of Cloudflare

I have a managed webserver with ~ 4000 images with 7,4 GB and I've enabled Cloudflare CDN for it.

I've run a curl script to download all images. I expected that the image will be put into Cloudflare (free plan) cache afterwards, but when I download them again via curl, it's still very slow. I can also see in the webinterface, that most requests are not cached.

enter image description here

Then I've run curl with -I to check the headers and I run it after a few minutes again and the result is totally different

$ cat curl-output-headers.txt | grep 'cf-cache-status: REVALIDATED'|wc
    303     606    9090
$ cat curl-output-headers.txt | grep 'cf-cache-status: MISS'|wc
    549    1098   12627
$ cat curl-output-headers.txt | grep 'cf-cache-status: HIT'|wc
   3267    6534   71874

#a few minutes later

$ cat curl-output-headers2.txt | grep 'cf-cache-status: HIT'|wc
    457     914   10054
$ cat curl-output-headers2.txt | grep 'cf-cache-status: MISS'|wc
   2428    4856   55844
$ cat curl-output-headers2.txt | grep 'cf-cache-status: REVALIDATED'|wc
   1234    2468   37020

When I download the files via GET, it takes ~15 minutes.

A short summary what I try to do: do a bulk upload to amazon seller via an excel file. It doesn't work because amazon tries to download too much images at the same time, even 100 rows. In my first attempt I was using images URLs by the manufacturer webserver. If I download them manually, it takes 7 minutes, but for amazon still to slow (but I guess amazon is downloading them in parallel, not like me in serial via curl).

Hoever, my idea was to use a CDN, but somehow it become even worse but I'm sure I'm doing something wrong.

I have also a VPS with a dedicated IP, maybe I should use that server?

Solution

  • Cloudflare doesn't guarantee every server response will always be cached, especially for a free account. With a free account they may not cache something until many requests for that resource start coming in.

    Also, by default your Cloudflare account will have a different cache on each Cloudflare edge location, which means if requests end up routed to different edge locations the cache will be different.

    You would need to enable Tiered Caching to fix the "multiple caches per edge" issue. And you would need to enable Cache Reserve to guarantee your server response is always cached.