wordpresspermalinksrobots.txt

wordpress robots


my permalink was like this %postname%/ then i changed it like this /%post_id%/%postname%/ now the problem is that google bot still indexing postname and report it as not found, so I want to add a line to robots.txt to prevent bots from indexing %postname%/ and only index /%post_id%/%postname%/ thanks


Solution

  • Not sure if this is really a question for SO but here are some points.

    1. You should 301 redirect from your old posts (there are plugins to do this, i cant remember the name from the top of my head)
    2. robots.txt will not fix this, the search engines hitting either a 404 or a 301 will cause them to drop your content from the index
    3. Even if you really want to block them using the robots.txt it will not cause the search engines to drop your content from the index, it will only stop them from accessing it in the future (and that if they conform to the robots.txt standard.

    Your best bet hear is to chase point no. 1, cause doing anything else will hurt your search engine rankings. Giving the search engines a 301 response tells them that your page has moved so they should update their links. You shouldn't loose ranking by using this method.

    Hope that helps.