[SOLVED] Create List of URLs for grequest

Create List of URLs for grequest

Having this multiple payloads dictionarys, how can one better create a list of formatted urls so grequest can itereate over ?

payload_single: {'search': '51 F ST SW,AUBURN KING 98001,WA,USA', 'app_code': 'xyz', 'app_id': 'xyz', 'lod': '9'}

payload_single: {'search': '55 F ST SW,AUBURN KING 98001,WA,USA', 'app_code': 'xyz', 'app_id': 'xyz', 'lod': '9'}

To be able to send requests via grequests iterating over URLS using the below code :

unsentrequests=(grequests.get(u, hooks = {'response' : do_something}) for u in urls) 
responses=grequests.map(unsentrequests)

For a single call to requests i would use this:

row = requests.get(url_single, params=payload_single)

EDIT 1 @rebeling

    for url in urls:
        unsent_request.append(grequests.get(url_single,
                                        hooks={'response': resphandler()},
                                        params=url))

def resphandler():
    rs = grequests.map(unsent_request)

    for r in rs:
        print r

Solution

import grequests  

def do_something(response, *args, **kwargs):  
    print response  


payloads = [
    {'search': '51 F ST SW,AUBURN KING 98001,WA,USA', 'app_code': 'xyz', 'app_id': 'xyz', 'lod': '9'},
    {'search': '55 F ST SW,AUBURN KING 98001,WA,USA', 'app_code': 'xyz', 'app_id': 'xyz', 'lod': '9'}]

unsent_request = []

See grequests code line 35: Accept same parameters as Session.request and some additional, so we use params as you would in requests.

for payload_single in payloads:
    unsent_request.append(grequests.get('http://www.google.com', 
                                        hooks={'response': do_something}, 
                                        params=payload_single))

print grequests.map(unsent_request)