pythonpython-2.7scraperwiki

Access data of a wiki scraper and store it in a local database

Hi i have written a scraper on scraper-wiki which scrape a web page and store the data in scraper-wiki database.Now i want to write a program in python which will go to scraper-wiki API and fetch the data and stores in a sq lite database of local machine.

Solution

  • Hey guys luckily i got the answer On scraper wiki there is an API here https://scraperwiki.com/docs/api#sqlite which will generate the URI of the table of a particular wiki scraper u want access of then u can open it as a url with the help of urllib module of python

    import urllib

def access_data():

f= urllib.urlopen('https://api.scraperwiki.com/api/1.0/datastore/sqlite?format=csv&name=example&query=select%20*%20from%20%60table%60')

data=f.read()
s = open('example.csv','w')
s.write(data)