I am trying to create an experiment on Amazon Mechanical Turk. I have borrowed the code from this SO question. It runs without an error, but returns an empty array. The code is:
import boto3
import xmltodict
MTURK_SANDBOX = 'https://mturk-requester-sandbox.us-east-1.amazonaws.com'
MTURK_PROD = 'https://mturk-requester.us-east-1.amazonaws.com'
mturk = boto3.client('mturk',
aws_access_key_id = "key",
aws_secret_access_key = "secret_key",
region_name='us-east-1',
endpoint_url = MTURK_SANDBOX
)
# Read in the questions.xml file saved in the same directory
question = open(file='questions.xml',mode='r', encoding='utf-8').read()
new_hit = mturk.create_hit(
Title = 'Map the following question to the paragraph/content where the answer to the question may lie.',
Description = 'Please review the following content to figure out where the answer to the posted question may reside',
Keywords = 'images, quick, labeling',
Reward = '0.00',
MaxAssignments = 1,
LifetimeInSeconds = 1728,
AssignmentDurationInSeconds = 600,
AutoApprovalDelayInSeconds = 14400,
Question = question,
AssignmentReviewPolicy={
'PolicyName':'ScoreMyKnownAnswers/2011-09-01',
'Parameters':[
{'Key':'AnswerKey', 'MapEntries':[{'Key': 'question_1', 'Values':['yes']}]},
{'Key': 'ApproveIfKnownAnswerScoreIsAtLeast', 'Values':['1']},
{'Key': 'RejectIfKnownAnswerScoreIsLessThan', 'Values':['1']},{'Key': 'RejectReason', 'Values':['Sorry, we could not approve your submission as you did not correctly identify the image containing the Flamingo.']},
{'Key': 'ExtendIfKnownAnswerScoreIsLessThan','Values':['1']}
]}
)
print("A new HIT has been created. You can preview it here:")
print(new_hit['HIT']['HITGroupId'])
print("https://workersandbox.mturk.com/mturk/preview?groupId=" + str(new_hit['HIT']['HITGroupId']))
# Retrive results
# Use the hit_id previously created
hit_id = new_hit['HIT']['HITId']
# We are only publishing this task to one Worker
# So we will get back an array with one item if it has been completed
worker_results = mturk.list_assignments_for_hit(HITId=hit_id, AssignmentStatuses=['Submitted'])
print(worker_results)
where questions.xml looks like:
<HTMLQuestion xmlns="http://mechanicalturk.amazonaws.com/AWSMechanicalTurkDataSchemas/2011-11-11/HTMLQuestion.xsd">
<HTMLContent><![CDATA[
<!-- YOUR HTML BEGINS -->
<!DOCTYPE html>
<html>
<head>
<meta http-equiv='Content-Type' content='text/html; charset=UTF-8'/>
<script type='text/javascript' src='https://s3.amazonaws.com/mturk-public/externalHIT_v1.js'></script>
</head>
<body>
<form name='mturk_form' method='post' id='mturk_form' action='https://www.mturk.com/mturk/externalSubmit'><input type='hidden' value='' name='assignmentId' id='assignmentId'/>
<h2>Is this Tweet happy, angry, excited, scared, annoyed or upset? Type in one word to describe the main emotion in the message. If it is unclear, type in "unclear".</h2>
<h3> Tweet: "I am really looking forward to the next Seahawks game!"</h3>
<div>
<input type='text' name='reported_emotion' placeholder='Type in your answer here'>
</div>
<p><input type='submit' id='submitButton' value='Submit' /></p></form>
<script language='Javascript'>turkSetAssignmentID();</script>
</body></html>
<!-- YOUR HTML ENDS -->
]]>
</HTMLContent>
<FrameHeight>600</FrameHeight>
</HTMLQuestion>
I then complete and. Accept the HIT using an MTurk worker account I created. The xml dictionary I get as worker_results is:
{'NumResults': 0, 'Assignments': [], 'ResponseMetadata': {'RequestId': '29d88641-ce06-4734-9fd4-b8ac8cb02d03', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': '29d88641-ce06-4734-9fd4-b8ac8cb02d03', 'content-type': 'application/x-amz-json-1.1', 'content-length': '33', 'date': 'Mon, 31 Aug 2020 17:37:51 GMT'}, 'RetryAttempts': 0}}
Shouldn't this be showing values in at least the first two entries?
Your code is working. The Assignments field will remain empty until a Worker has accepted and submitted your HIT.