I'm trying to understand the behavior of HTTP referer header. I noticed that sometimes the referer is full (full URL, including path and query string) but mostly it includes the domain only.
For example 'https://www.google.com/' instead of 'https://www.google.com/search?q=http+referer+truncated&oq=http+referer+truncated&aqs=chrome..69i57.6485j0j1&sourceid=chrome&ie=UTF-8#q=http+referer+is+not+full'
Are there any rules as to when the refere and is full and when it's truncated?
HTTP referrer headers are created by browsers according to desired criteria using Referrer Policy even though there is a general standard used by majority of the browsers there are some differences about how the browsers handles the servers instructions, mainly mobile web browsers are the ones which does not cooperate nicely with WWWC recommendations on this matter.
So why is there need for different HTTP referrer headers? To understand this we need to look at what are these headers are used for first. Main purpose in its simplest form is "carrying information from the originating page to the new page".
Everywhere we see the word "information" in the web there is a information security concept attached to it and HTTP header is no different. Depending on what kind of information headers carry, server can specify the type of referrer policy needs to be used. Here is the list of referrer policies from W3
enum ReferrerPolicy {
"",
"no-referrer",
"no-referrer-when-downgrade",
"same-origin",
"origin",
"strict-origin",
"origin-when-cross-origin",
"strict-origin-when-cross-origin",
"unsafe-url"
};
Detailed information about each of these are available in the Referrer policy link i included above.
To give an example; Using google searching for "Yellow Pages". in this case
origin:https://www.google.ie
referer:https://www.google.ie/
Referrer Policy:origin
generated URL:https://www.google.ie/gen_204?atyp=i&ct=&cad=udla=3&ei=x65kGDkdyKGHDkF0KeoBg&e=12&zx=1494785478502
link to the first result is
While the actual URL is https://www.goldenpages.ie/
When we actually click the link referrer changes to
Referer:https://www.goldenpages.ie/ and the referrer policy is
Referrer Policy:no-referrer-when-downgrade
This means if we click another link from the current page we won't see all the additional parameters similar to the ones we saw in the URL from google search results page.
To prove this is the case; click any link from the current page and watch the referrer header changing according to the the policy type (Which can be found in the associated js file if you use developer tools and inspect the network activity)
When i click the "List your business" link referrer stay as
https://www.goldenpages.ie/list-your-business/
and no other parameters are passed
So just to tidy up this messy explanation; What URL gets generated is dependent on what rules are set regarding to Referrer policy may that be a simple base rule with no parameters or a very long URL with loads of information relating to the user and origin of the navigation.
Note: URLs wont work i have jumbled some letters.