rubyperformanceamazon-s3storagefog

ruby-fog: Delete an item from the object storage in less than 3 requests


I started using fog storage for a project. I do the most simple actions: upload an object, get the object, delete the object. My code looks something like this:

storage = get_storage(...) // S3 / OpenStack / ...
dir = storage.directories.get(bucket) # 1st request
if !dir.nil?
  dir.files.create(key: key, body: body) # 2nd request

  # or:
  dir.files.get(key) # 2nd request

  #or
  file = dir.files.get(key) # 2nd request
  if !file.nil?
    file.destroy # 3rd request
  end
end

In all cases there's a 1st step to get the directory, which does a request to the storage engine (it returns nil if the directory doesn't exists).

Then there's another step to do whatever I'd like to do (in case of delete there's even a 3rd step in the middle).

However if I look at let's say the Amazon S3 API, it's clear that deleting an object doesn't need 3 requests to amazon.

Is there a way to use fog but make it do less requests to the storage provider?


Solution

  • I think this was already answered on the mailing list, but if you use #new on directories/files it will give you just a local reference (vs #get which does a lookup). That should get you what you want, though it may raise errors if the file or directory does not exist.

    Something like this:

    storage = get_storage(...) // S3 / OpenStack / ...
    dir = storage.directories.new(key: bucket)
    
    dir.files.create(key: key, body: body) # 1st request
    
    # or:
    dir.files.get(key) # 1st request
    
    #or
    file = dir.files.new(key)
    
    if !file.nil?
      file.destroy # 1st request
    end
    

    Working in this way should allow any of the 3 modalities to work in a single request, but may result in errors if the bucket does not exist (as trying to add a file to non-existent bucket is an error). So it is more efficient, but would need different error handling. Conversely, you can make the extra requests if you need to be sure.