rubyweb-crawleranemone

Crawling sub-domain with Anemone

I am using Anemone. How do I crawl sub-domain too? for e.g if I have website www.abc.com my crawler should also crawl support.abc.com or blah.abc.com. I am using Ruby 1.8.7 and Rails 3.

Solution

  • Here is a commit on Github that solves your problem.

    https://github.com/runa/anemone/commit/91559bde052956cfc40ae62678ec2a61574cf928

    Change your anemone gem files as per the link.