seoopencartrobots.txtmultistore

How do i tell google to not crawl a domain completly

I have a site in opencart say abc.com and i have opened a multi store with it xyz.com and i have found that google has started crawling xyz.com too which i don't want . Both the domains are pointing to the same directory so i suppose there can only be one robots.txt file . I am not very experienced with google webmaster or SEO so please tell me how can i tell google to ignore all the urls starting with xyz.com .

Solution

  • If you are using Apache and mod_rewrite you can add a rewrite rule to serve a different robots.txt file for xyz.com:

    RewriteCond %{HTTP_HOST} xyz.com$ [NC]
RewriteRule ^robots.txt robots_xyz.txt [L]

    Then create robots_xyz.txt:

    User-agent: *
Disallow: /