I am writing a Web scraper using Perl to fetch data from the http://www.coupons.com/ The problem is that the HTML I fetch with LWP::UserAgent is different from what I see in the web browser. I am interested in the content of the JavaScript variable "CouponClubMember" and in HTML I receive with Perl this variable is empty.
Any ideas?
Using code below, I am getting same thing as with my browser. I just set agent
to same string my Firefox sent and enabled cookie handling:
use LWP::UserAgent;
my $ua = LWP::UserAgent->new(
agent => 'Mozilla/5.0 (Windows NT 5.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1',
cookie_jar => {},
);
$res = $ua->get("http://www.coupons.com");
if($res->content =~ /(CouponClubMember.{300})/) {
print $1;
}