How to investigate repeated cached lambda at edge requests

I have the following Cloudfront and lambda at edge setup

• User makes a request to CDN (path is irrelevant, result must always be the same for given day)
• Cloudfront makes request to origin (lambda is invoked to determine which s3 resource to fetch)
• Cloudfront receives response (lambda is invoked to write Expires header in UTC)
• User receives response

Visually it would be like this

fig 1. Visual CDN architecture

In my behavior cache settings I have the following set:

• Min TTL 0
• Max TTL 86400 (24 hours)
• Default TTL 0
• Legacy settings to prevent including cookies, query strings, and headers from busting the cache

Given that cloudfront caches everything per region, and that I do not expect many hits on the CDN at all (5 a day at most), I'd expect only one origin request to be made per day per region, but sometimes I see 2 requests being made hours apart for the same day

fig 2. Cloudfront monitoring for origin requests

Question stands: Am I misunderstanding how caching works in cloudfront, considering I write Expires end of day header (for april 29th I'd write 2025-04-30 00:00:00 UTC) and this is expected behavior, or is there a misconfiguration somewhere within caching? Are there tools to test caching configurations?

You’re mostly on the right track, but there are a few nuances with how CloudFront handles caching and Lambda@Edge that could explain the behavior you’re seeing.

What’s happening

Even if you:

• Set Expires to the end of the UTC day (e.g. 2025-04-30 00:00:00)
• Have TTLs like Default TTL = 0, Min TTL = 0, and Max TTL = 86400
• Remove query strings, cookies, and headers from cache keys

You can still see multiple origin fetches per day per edge location. This is because:

• CloudFront caches are distributed across edge locations (i.e., regional PoPs), and each maintains its own cache.
• When a user from a new region hits your distribution, CloudFront might invoke your Lambda@Edge and forward the request to origin since that edge node hasn’t cached it yet.
• A cold cache in a different location = new origin request = new Lambda@Edge invocation.

Why this can happen

Your graph from the Lambda@Edge metrics shows multiple low-volume invocations. This likely means:

• A few users in different regions made requests.
• Cache in each region was cold (i.e., TTL expired or not set yet).

This is expected behavior for a low-traffic CDN setup.

Tips to reduce this

• Set a longer Default TTL (e.g. 300 or 600 seconds) if the content doesn’t change mid-day. The Expires header is respected by browsers, but CloudFront uses its own TTL settings.
• Enable Origin Shield in CloudFront to reduce duplicate origin fetches by funneling all regional misses through a central location.
• Log viewer IPs and X-Amz-Cf-Region headers temporarily (if allowed) to verify if requests are coming from different edge regions.

Tools to debug

• CloudFront logs: Enable standard logging to S3 or use real-time logs for edge-level insights.
• Use curl -I with --resolve to simulate a request from different locations (or CloudShell across regions).
• Lambda@Edge logs via CloudWatch can show the region and request context.

TL;DR:

CloudFront caching is per edge location, not global. Low traffic across multiple regions can cause multiple origin fetches—even with proper Expires and TTL settings. Use Origin Shield and log tools to better visualize and reduce these edge cache misses.