I am ultimately trying to generate a csv from the JSON output of a Monday.com API request.
Below is my code so far. I am having trouble trying to flatten the JSON into a table.
import requests
import json
import pandas as pd
apiKey = "API Key Here"
apiUrl = "https://api.monday.com/v2"
headers = {"Authorization" : apiKey}
query2 = '{boards(ids:123456) {items{name, column_values{title text } } } }'
data = {'query' : query2}
json_data = json.loads(requests.post(url=apiUrl, json=data, headers=headers).text)
norm=pd.json_normalize(json_data, 'items',['data', 'boards'])
JSON Output from the API. I added some linebreaks for human readability.
{'data':
{'boards':
[{'items':
[{'name': 'Item 1', 'column_values': [{'title': 'Person', 'text': 'Mark McCoy'}, {'title': 'Status', 'text': None}, {'title': 'Date', 'text': '2021-02-05'}]},
{'name': 'This is a new item', 'column_values': [{'title': 'Person', 'text': ''}, {'title': 'Status', 'text': None}, {'title': 'Date', 'text': '2021-04-17'}]},
{'name': 'Item 5', 'column_values': [{'title': 'Person', 'text': ''}, {'title': 'Status', 'text': None}, {'title': 'Date', 'text': '2021-02-13'}]},
{'name': 'Item 2', 'column_values': [{'title': 'Person', 'text': ''}, {'title': 'Status', 'text': 'Done'}, {'title': 'Date', 'text': '2021-05-14'}]}]}]},
'account_id': 00000000}
When I run the ,py file in terminal I get the below output
Traceback (most recent call last):
File "/Users/markamccoy/Desktop/MondayPy/stack.py", line 14, in <module>
norm=pd.json_normalize(json_data, 'items',['data', 'boards'])
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/json/_normalize.py", line 336, in _json_normalize
_recursive_extract(data, record_path, {}, level=0)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/json/_normalize.py", line 309, in _recursive_extract
recs = _pull_records(obj, path[0])
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/json/_normalize.py", line 248, in _pull_records
result = _pull_field(js, spec)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/site-packages/pandas/io/json/_normalize.py", line 239, in _pull_field
result = result[spec]
KeyError: 'items'
I am total python noob and reading the pandas documentation has gotten me this far but I am a little out of my element.
If you use normalize.the result will looks like this.
df = pd.json_normalize(json_data['data']['boards'][0]['items'],record_path='column_values',meta=['name'])
title text name
0 Person Mark McCoy Item 1
1 Status None Item 1
2 Date 2021-02-05 Item 1
3 Person This is a new item
4 Status None This is a new item
5 Date 2021-04-17 This is a new item
But I think is not what you want.and there is no example to flat that kinds of array in json_normalize.
data = [ [item['name']]+[c_v['text'] for c_v in item['column_values']] for item in json_data['data']['boards'][0]['items']]
df = pd.DataFrame(data,columns=['name','Person','Status','Date'])
name Person Status Date
0 Item 1 Mark McCoy None 2021-02-05
1 This is a new item None 2021-04-17
2 Item 5 None 2021-02-13
3 Item 2 Done 2021-05-14
so I can only flat it in python.
if you have futher question.please comment to me