resthadoophbasenutchstargate

How to lookup HBase REST API (Stargate) if the row-keys are reversed urls

I am using nutch2.2.1 + hbase0.90.4, and wanting to access the data via the HBase REST API Stargate. If I seed nutch with a url (eg. www.usatoday.com), the reversed url becomes the HBase row-key in the designated table ('webpage'). I can lookup the data via the hbase shell as follows: 

hbase(main):001:0> get 'webpage', 'com.usatoday.www:http/'
COLUMN                                         CELL                                                                                                                                 
 f:fi                                          timestamp=1404762373394,value=\x00'\x8D\x00                                                                                         
 f:ts                                          timestamp=1404762373394, value=\x00\x00\x01G\x12\\xB5\xB3                                                                            
 mk:_injmrk_                                   timestamp=1404762373394, value=y                                                                                                     
 mk:dist                                       timestamp=1404762373394, value=0                                                                                                     
 mtdt:_csh_                                    timestamp=1404762373394, value=?\x80\x00\x00                                                                                         
 s:s                                           timestamp=1404762373394, value=?\x80\x00\x00

However, I am having trouble using the REST API. Presumably I need to do some pretty simple URL encoding to suppress the colon before 'http' that is making trouble for me?

For eg., I get a HTTP 404 when I try

curl http://localhost:8900/webpage/com.usatoday.www:http/

also when I try

curl http://localhost:8900/webpage/com.usatoday.www%3Ahttp/

I know that the REST API is working fine as I can create a row called 'row3' into a table called 'test' and lookup

curl http://localhost:8900/test/row3

to see the following expected result: 

<?xml version="1.0" encoding="UTF-8" standalone="yes"?><CellSet><Row key="cm93Mw=="><Cell timestamp="1404761922130" column="Y2Y6Yw==">dGhpcyBpcyBzb3J0YSB3b3JraW5nIG5vdw==</Cell></Row></CellSet>

Thanks for the help!

Solution

  • Needed to URL encode the forward slash as well. The following works.

    curl http://localhost:8900/webpage/com.usatoday.www%3Ahttp%2F