I have been learning how to use Amazon S3 API by using the open source Eucalyptus. So far I have been able to successfully use REST, but now I would also like to use SOAP. I seem to be having trouble generating the correct signature for my request. The service is giving me a 403 Forbidden error:
Traceback (most recent call last):
File "soap.py", line 31, in <module>
r = w.download_file('mybucket', 'test.txt')
File "soap.py", line 27, in download_file
r = self.client.service.ListAllMyBuckets(self.access_key, timestamp, signature)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 521, in __call__
return client.invoke(args, kwargs)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 581, in invoke
result = self.send(soapenv)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 619, in send
description=tostr(e), original_soapenv=original_soapenv)
File "/usr/lib/python2.6/site-packages/suds/client.py", line 677, in process_reply
raise Exception((status, description))
Exception: (403, u'Forbidden')
My code is in Python 2 and uses the SUDS-Jurko library for sending SOAP requests:
from suds.client import Client
class WalrusSoap:
wsdl_url = 'https://s3.amazonaws.com/doc/2006-03-01/AmazonS3.wsdl'
server_url = 'https://localhost:8773/services/Walrus'
def __init__(self, access_key, secret_key):
self.access_key = access_key
self.secret_key = secret_key
self.client = Client(self.wsdl_url)
self.client.wsdl.services[0].setlocation(self.server_url)
#print self.client
def create_signature(self, operation, timestamp):
import base64, hmac, hashlib
h = hashlib.sha1(self.secret_key)
h.update("AmazonS3" + operation + timestamp)
#h = hmac.new(key=self.secret_key, msg="AmazonS3" + operation + timestamp, digestmod=hashlib.sha1)
return base64.encodestring(h.digest()).strip()
def download_file(self, bucket, filename):
from time import gmtime, strftime
timestamp = strftime('%Y-%m-%dT%H:%M:%S.001Z', gmtime())
print(timestamp)
signature = self.create_signature('ListAllMyBuckets', timestamp)
print(signature)
r = self.client.service.ListAllMyBuckets(self.access_key, timestamp, signature)
return r
w = WalrusSoap(access_key='MOBSE7FNS6OC5NYC75PG8', secret_key='yxYZmSLCg5Xw6rQVgoIuVLMAx3hZRlxDc0VOJqox')
r = w.download_file('mybucket', 'test.txt')
print(r)
I changed the server endpoint, because otherwise the WSDL points to the regular S3 servers at Amazon. I also have two different ways of creating the signature in my create_signature function. I was swapping between one and the other by simply commenting out the second one. Neither of the two seem to work. My question is what am I doing wrong?
SOAP Authentication: http://docs.aws.amazon.com/AmazonS3/latest/dev/SOAPAuthentication.html
SUDS-Jurko Documentation: https://bitbucket.org/jurko/suds/overview
Edit: I realized I forgot to include an example of what timestamp and signature is printed for debugging purposes.
2014-12-05T00:27:41.001Z
0h8vxE2+k10tetXZQJxXNnNUjjw=
Edit 2: Also, I know that the download_file function does not download a file :) I am still in testing/debug phase
Edit 3: I am aware that REST is better to use, at least according to Amazon. (Personally I think REST is better also.) I am also already aware that SOAP is deprecated by Amazon. However I would like to go down this path anyways, so please do me a favor and do not waste my time with links to the deprecation. I assure you that while writing this SOAP code, I was already well aware of the deprecation. In fact one of the links I posted has the deprecation notice printed at the top of its page. However, if you have evidence showing that Walrus completely ditches SOAP or that they stopped working on the SOAP portion, I would like to see something like that. But please do not tell me Amazon has deprecated SOAP.
Eucalyptus does support SOAP (see ZachH's comment about deprecation): https://www.eucalyptus.com/docs/eucalyptus/4.0.2/schemas/aws-apis/s3/index.html
I decided to scrap using python, and I produced some working code with C# instead. Turns out C# has better SOAP libraries.