pythonweb-scraping

Save json request to a file


thanks for your attention!

I'm doing a for loop and "requesting url" here on python and getting a "json" response, want to add all the loops to a file to read later on excel, or other json reader...

for x in range(1, 6):
    url = f"https://api.xxx.com.br/ecommerce-web/api/parceiro/meuCadastro/{x}/true"
    response = requests.get(url, headers=headers)
    print(response.json())

So after this loop I got 5 responses like this

{'A': 1, 'Name': 'Name01'}
{'A': 2, 'Name': 'Name02'}
{'A': 3, 'Name': 'Name03'}
{'A': 4, 'Name': 'Name04'}
{'A': 5, 'Name': 'Name05'}

And i want to do a json file with all that answers, thank you guys! I'm just starting on webscraping.

Solve this problem add all the answers on json file


Solution

  • You could use the built-in json library to write json files.

    import json
    
    data = []  # save all responses here
    
    for x in range(1, 6):
        url = f"https://api.xxx.com.br/ecommerce-web/api/parceiro/meuCadastro/{x}/true"
        response = requests.get(url, headers=headers)
        data.append(response.json())  # append every json response
    
    
    with open("myjson.json", "w") as file:  # "w" will create myjson.json if it doesn't exist, otherwise rewrite the whole file
        json.dump(data, file, indent=4)  # dump all the data into the json file with 4 space indentation