erlangmochiweb

Parsing the result obtained from mochiweb_html

I would like to parse some content from an html file (no xml).

At the moment I retrieve the structure to parse using mochiweb_html: 

1> inets:start().
2> {ok, {Status, Headers, Body}} = httpc:request("http://www.google.com").
3> {String, Attributes, Other} = mochiweb_html:parse(Body).

and the result is something like:

{<<"html">>,
 [{<<"itemscope">>,<<"itemscope">>},
  {<<"itemtype">>,<<"http://schema.org/WebPage">>}],
 [{<<"head">>,[],
   [{<<"meta">>,
     [{<<"itemprop">>,<<"image">>},
      {<<"content">>,<<"/images/google_favicon_128.png">>}],
     []},
    {<<"title">>,[],[<<"Google">>]},
....

What is the best way to retrieve from a structure obtained from mochiweb_http all the elements in the web page that have a specific tag with a specific class (e.g., <span id="footer">)?

Solution

  • You could use mochiweb_xpath:

    > mochiweb_xpath:execute("//span[@id='footer']",
    mochiweb_html:parse(
      "<html><body><span>not this one</span><span id='footer'>but this one</span></body></html>")).
[{<<"span">>,
  [{<<"id">>,<<"footer">>}],
  [<<"but this one">>]}]