CakePHP themed with robots.txt

I have a CakePHP application that uses themed views, css and images. Now I need to have themed robots.txt as well.

Naturally I put a robots.txt file in every theme folder /View/Themed/Theme/webroot/robots.txt, but only the one in the app/webroot gets displayed. Usually themed files (img,css,js,ctp) will overwrite the "default" files.

Is there a way to have different robots.txt files?

note: before I used to have something like this in the htaccess file

RewriteCond %{HTTP_HOST} ^bla.website.de$
RewriteRule robots.txt bla.robots.txt [L]

RewriteCond %{HTTP_HOST} ^foo.website.de$
RewriteRule robots.txt foo.robots.txt [L]

RewriteCond %{HTTP_HOST} !^www.website.de$
RewriteRule robots.txt noindex.robots.txt [L]

But this will get out of hand very fast.

Themes don't work like that

Usually themed files (img,css,js,ctp) will overwrite the "default" files.

Actually, that's not the case. CakePHP can only handle requests that reach the php code, if there is a static file matching the path in the way, that will get served directly by the webserver, as that's what the default rewrite rules implement.

When using a theme CakePHP will change the requested url i.e.:

app/Plugin/DebugKit/webroot/js/my_file.js becomes app/webroot/debug_kit/js/my_file.js

app/View/Themed/Navy/webroot/css/navy.css becomes app/webroot/theme/Navy/css/navy.css

So it is not that the response for assets (/css/somefile.css) is modified, an entirely different request (/using_this_theme/css/somefile.css) is made when using a theme.

This is, in fact, something you're expected to take advantage of, by putting or linking your theme assets directly in the webroot as serving files via php is slower than not invoking php.

Use a route and a controller action, not rewrite rules

Since you have a multiple-domain-points-at-one-application setup (implied) and want a single url (/robots.txt) to return different content it's not possible with the default rewrite rules to make the url point at a static file¹. To make the contents of /robots.txt dynamic, simply delete the file in webroot/robots.txt and add a route and controller action to handle serving the content based on the theme that's relevant to the request.

¹ Obviously it is possible to do it by changing rewrite rules, but that means added complexity, and prevents the response being configurable (using a different theme -> instant change of content). You could consider generating your rewrite rules programatically though