pythonmimeencodemultipartform-dataposter

Problems using multipart_encode (poster library)


I am trying to upload a file using multipart_encode to realize the MIME process. However, I met the following error AttributeError: multipart_yielder instance has no attribute '__len__'. Below are is my approach, I really appreciate if anyone can give me some suggestions.

url = "https://pi-user-files.s3-external-1.amazonaws.com/"           
post_data = {}
#data is a dict
post_data['AWSAccessKeyId']=(data['ticket']['AWSAccessKeyId'])
post_data['success_action_redirect']=(data['ticket']['success_action_redirect'])
post_data['acl']=(data['ticket']['acl'])
post_data['key']=(data['ticket']['key'])
post_data['signature']=(data['ticket']['signature'])
post_data['policy']=(data['ticket']['policy'])
post_data['Content-Type']=(data['ticket']['Content-Type'])

#I would like to upload a text file "new 2"
post_data['file']=open("new  2.txt", "rb")

datagen, headers = multipart_encode(post_data)
request2 = urllib2.Request(url, datagen, headers)
result = urllib2.urlopen(request2)

Solution

  • The problem is that in httplib.py, the generator is not detected as such and is treated instead like a string that holds the full data to be sent (and therefore it tries to find its length):

    if hasattr(data,'read') and not isinstance(data, array): # generator 
        if self.debuglevel > 0: print "sendIng a read()able"
        ....
    

    A solution is to make the generator act like a read()able:

    class GeneratorToReadable():
        def __init__(self, datagen):
            self.generator = datagen
            self._end = False
            self.data = ''
    
        def read(self, n_bytes):
            while not self._end and len(self.data) < n_bytes:
                try:
                    next_chunk = self.generator.next()
                    if next_chunk:
                        self.data += next_chunk
                    else:
                        self._end = True
                except StopIteration:
                    self._end = True
            result = self.data[0:n_bytes]
            self.data = self.data[n_bytes:]
            return result
    

    and use like so:

    datagen, headers = multipart_encode(post_data)
    readable = GeneratorToReadable(datagen)
    req = urllib2.Request(url, readable, headers)
    result = urllib2.urlopen(req)