How to directly get the output of subprocess.check_output as valid json rather than reading the results stored in a file?

I have a system command which produces json output: ($ cmd -J > file). I can read this data perfectly well into a dict using:

with open("file", "r") as i:
    data=json.loads(i)

I'd like to do the same thing without the intermediary file, directly generating the data via a subprocess call as:

data=subprocess.check_output(["cmd", "-J"]).decode("utf-8")

or:

call=subprocess.call(["cmd", "-J"], capture_output=True)
data=call.output.decode("utf-8")

but json.dumps(data) does not produce a dict, there are \\" and \\n characters everywhere. Calling replace to remove them does not improve the situation.

How can I get the same dict as the one I got simply reading the command's output stored as a file on disk?

Solution

is something along the following what you are after ?

cat ~/tmp/test.py
import subprocess
import json

#sample json data
command = ['curl', '-s',  'https://microsoftedge.github.io/Demos/json-dummy-data/64KB.json']
res = subprocess.run(command, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True)

if res.returncode == 0:
    try:
        jsonOut = json.loads(res.stdout)
        for item in range(5):
            print(jsonOut[item],'\n')
    except json.JSONDecodeError as e:
        print("Failed to parse:", e)
else:
    print("Error:", res.stderr)

#run it
python /tmp/test.py

#produces...
{'name': 'Adeel Solangi', 'language': 'Sindhi', 'id': 'V59OF92YF627HFY0', 'bio': 'Donec lobortis eleifend condimentum. Cras dictum dolor lacinia lectus vehicula rutrum. Maecenas quis nisi nunc. Nam tristique feugiat est vitae mollis. Maecenas quis nisi nunc.', 'version': 6.1} 

{'name': 'Afzal Ghaffar', 'language': 'Sindhi', 'id': 'ENTOCR13RSCLZ6KU', 'bio': 'Aliquam sollicitudin ante ligula, eget malesuada nibh efficitur et. Pellentesque massa sem, scelerisque sit amet odio id, cursus tempor urna. Etiam congue dignissim volutpat. Vestibulum pharetra libero et velit gravida euismod.', 'version': 1.88} 

{'name': 'Aamir Solangi', 'language': 'Sindhi', 'id': 'IAKPO3R4761JDRVG', 'bio': 'Vestibulum pharetra libero et velit gravida euismod. Quisque mauris ligula, efficitur porttitor sodales ac, lacinia non ex. Fusce eu ultrices elit, vel posuere neque.', 'version': 7.27} 

{'name': 'Abla Dilmurat', 'language': 'Uyghur', 'id': '5ZVOEPMJUI4MB4EN', 'bio': 'Donec lobortis eleifend condimentum. Morbi ac tellus erat.', 'version': 2.53} 

{'name': 'Adil Eli', 'language': 'Uyghur', 'id': '6VTI8X6LL0MMPJCC', 'bio': 'Vivamus id faucibus velit, id posuere leo. Morbi vitae nisi lacinia, laoreet lorem nec, egestas orci. Suspendisse potenti.', 'version': 6.49}