perlurlhttp-redirectwww-mechanizelwp

Perl WWW::Mechanize (or LWP) get redirect url


So I am using WWW::Mechanize to crawl sites. It works great, except if I request a url such as:

http://www.levi.com/

I am redirected to:

http://us.levi.com/home/index.jsp

And for my script I need to know that this redirect took place and what the url I was redirected to is. Is there anyway to detect this with WWW::Mechanize or LWP and then get the redirected url? Thanks!


Solution

  • use strict;
    use warnings;
    use URI;
    use WWW::Mechanize;
    
    my $url = 'http://...';
    my $mech = WWW::Mechanize->new(autocheck => 0);
    $mech->max_redirect(0);
    $mech->get($url);
    
    my $status = $mech->status();
    if (($status >= 300) && ($status < 400)) {
      my $location = $mech->response()->header('Location');
      if (defined $location) {
        print "Redirected to $location\n";
        $mech->get(URI->new_abs($location, $mech->base()));
      }
    }
    

    If the status code is 3XX, then you should check response headers for redirection url.