Is the following valid in my robots.txt file?
Disallow: /*?action=addwishlist&product_id=*
rather than writing individually for every product like below:
Disallow: /*?action=addwishlist&product_id=1212
Or do I need to define below line for each individual products?
Disallow: /*?action=addwishlist&product_id=1212
Disallow: /*?action=addwishlist&product_id=1213
Disallow: /*?action=addwishlist&product_id=1214
or *
sign will be taken as any product id. product_id=*
Googlebot recognizes the asterisk in place of the product ID as tested with their robots.txt test tool:
You could also leave the asterisk off, because robots.txt
rules are all "starts with" rules. An asterisk at the end of the rule is equivalent to the same rule without the asterisk.
Keep in mind that while the major search engines support wildcards in Disallow
rules, other bots won't. Wildcards in disallow were not part of the original robots.txt specification from robotstxt.org. Less sophisticated crawlers that don't understand these wildcards will still crawl these URLs.