[SOLVED] java IOException when parsing a URL of a page for creating a DOM using TagSoup

java IOException when parsing a URL of a page for creating a DOM using TagSoup

Usng the following link I am trying to create a DOM tree of the URL (thats a specific URL that returns this exception) :

     String url="http://www.kingfisher.org/";

    Parser p = new Parser();
    SAX2DOM sax2dom ;
    org.w3c.dom.Node doc ;

    p.setFeature(Parser.namespacesFeature, false);
    p.setFeature(Parser.namespacePrefixesFeature, false);
    sax2dom = new SAX2DOM(true);
    p.setContentHandler(sax2dom);
    p.parse(new InputSource(url));
    doc = sax2dom.getDOM();

but whn I run my program for this URL, it gives me the excepton at p.parse(new InputSource(url)); which I dont know why. cause so far it had no problem whatsoever.

Exception in thread "main" java.io.IOException: Server returned HTTP response code: 403 for URL: http://www.kingfisher.org/
at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1838)
at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1439)
at org.ccil.cowan.tagsoup.Parser.getInputStream(Parser.java:510)
at org.ccil.cowan.tagsoup.Parser.getReader(Parser.java:487)
at org.ccil.cowan.tagsoup.Parser.parse(Parser.java:440)
at pageparsertest.PageParserTest.main(PageParserTest.java:92)

any hint?

Solution

if you use HttpURLConnection you should be able to access the request of web page from java. Try the following code:

String url = "http://www.kingfisher.org/";
        URL uri = new URL(url);
        HttpURLConnection httpcon = (HttpURLConnection) uri.openConnection();
        httpcon.addRequestProperty("User-Agent", "Mozilla/4.76");
        p.parse(new InputSource(httpcon.getInputStream()));