When I "view source" (using cmd-I )of google.com/robots.txt in Safari, Chrome or Firefox on a Mac, I see:
<html>
<head></head>
<body>
<pre style="word-wrap: break-word; white-space: pre-wrap;">…</pre>
</body>
</html>
Where is the HTML coming from?
Google's robots file is just an example -- I get the same results for all text files, in all browsers.
I am asking because I am writing a Python program to generate robots.txt and sitemap.txt files, and I wanted to make sure the correct headers were sent.
Seeing HTML in the source code made me worry that my headers were incorrect.
The fact that Google's robots.txt file has the same HTML reassures me that it's normal, but I would like to know why this is the case.
I have been building web sites since 1995 and do not remember ever noticing this behavior.
The HTML is generated in your Browser for viewing it. It's not sent this way (use e.g. curl to see it or the network analysis function of your Browser).
E.g. in Chrome: Menu -> More Tools -> Developer Tools. Then click on the tab "Network" and refresh the page.