Custom marker missing ... sometimes

We're using the static google maps API and things were going fine when suddenly it started getting hiccups with our custom marker icon. Sometimes it get's displayed as intended, sometimes we end up with the default marker and the yellow banner in the upper right corner stating: "Map error: g.co/staticmaperror".

Taking a look at the headers we found that google seems to have issues fetching our marker image:

x-staticmap-api-warning: Failed to fetch image url https://www.xyz.de/some_image.png

I can fire the very same request n times and get some m <= n responses that are fine and some o <= n requests that have the above mentioned issue. This seems to me like google has some issues caching the image.
I have no issues requesting the marker image from our servers, though. There are no firewall rules that fire. The marker image has a high cache-control max-age and a fine etag, it expires two weeks from now. There's nothing fancy about it.

The maps request looks like this:

https://maps.googleapis.com/maps/api/staticmap?key=<OUR_KEY>&size=260x130&maptype=roadmap&format=png&visual_refresh=true&zoom=10&markers=icon:https://www.xyz.de/some_image.png|52.52008961%2C13.40465566&signature=<SIG>

Nothing fancy here either.

Since SO seems to be the official google support now, may some google techie please help?

As it turns out our application firewall actually blocked some requests from Google and we have to add some more rules. (Although we think we checked that twice already.)

Sadly, there's no fixed IP range for the maps services. But let me share with you what the support told me:

---

Google Cloud Support

It is not possible to provide a specific list of IP addresses that must be accessible in order to use the Maps APIs, because Google services are not tied to specific hosts or IP addresses in our data centers.

Services are migrated between data centers depending on load, maintenance schedules, and growth. The particular data center that a specific request is routed to depends on the user's geographical location, network location, and the relative load on nearby data centers currently hosting the service concerned, and on the network links that connect them.

If you need to add IP based firewall rules, your only option is to whitelist Google's entire network allocation, which can be obtained using the following command from a Unix system:

$ dig +short _spf.google.com txt

The answer will include some text like:

"v=spf1 include:_netblocks.google.com include:_netblocks2.google.com include:_netblocks3.google.com ~all"

Be sure to follow the includes for all the netblocks e.g.

$ dig +short _netblocks.google.com txt

$ dig +short _netblocks2.google.com txt

$ dig +short _netblocks3.google.com txt

The answer section lists all IP address blocks in CIDR notation.

This list changes several times a year as servers are added and removed, and it is your responsibility to track those changes and update your firewall accordingly. (We recommend that you do so every quarter).

In general it is much easier to set firewall rules based on the list of hostnames used by the Maps API, which is more stable.

---

Another way to fix this would be to make an exception for this static resource (the marker icon).