rubysqliterubygemsscraperwiki

A table of scraperwiki.sqlite isn't found


I have a script in Ruby which uses scraperwiki gem. In the directory of this script there's file titled scraperwiki.sqlite.

items.each do |x|
   if ScraperWiki.select("* from data where .... { x['key123']}'").empty? 
     ScraperWiki.save_sqlite(['key123'], x)
   else
    puts "Skipping already saved record " + record['key123']
   end
end

But nonetheless when I run it I get an error:

/Users/alex/.rvm/gems/ruby-2.1.2/gems/sqlite_magic-0.0.3/lib/sqlite_magic.rb:49:in `rescue in execute': no such table: data (SqliteMagic::NoSuchTable)
  from /Users/alex/.rvm/gems/ruby-2.1.2/gems/sqlite_magic-0.0.3/lib/sqlite_magic.rb:42:in `execute'
  from /Users/alex/.rvm/gems/ruby-2.1.2/gems/scraperwiki-3.0.2/lib/scraperwiki.rb:186:in `select'

Solution

  • Two things:

    1. The ScraperWiki gem does not create the data (actually swdata) table until you've saved some data
    2. Once you have saved some data, the table appears to be called swdata, not data

    Note: swdata is the default but you can change the table_name to data using arguments/configuration variables.

    Also note: this create-on-save (not-on-query) is not necessarily part of ScraperWiki's gem and is instead the way it's dependency of https://github.com/openc/sqlite_magic works. To see for yourself, look at the code at https://github.com/openc/sqlite_magic/blob/master/lib/sqlite_magic.rb and see how #create_table is only called from #insert_or_update and #save_data