pythonjsonpandasdataframe

Create JSON from CSV and add some header lines with Pandas


I found this post which initially seemed to be exactly what I was looking for but it didn't help me: Adding Header and Footer to JSON output from Python

I have a csv file which I read in as Pandas dataframe:

import os
import csv
import json
import pandas as pd
csvFilePath = "Mypath"
track = pd.read_csv(csvFilePath, header = 0, skiprows = 0, delimiter = ";")

The example csv looks like this:

Param1;Param2;name;lat;lon;altitude;vert_rate;track;speed;category;Param3;Param4;Param5
999999;9999;rocket;57.878696;11.160667;1089;0;137;2;99;999;16;0
999999;9999;rocket;57.878796;11.160668;2543.963336;152638.0483;137;2;99;999;15;0
999999;9999;rocket;57.878896;11.160670;4226.050004;126781.7063;137;2;99;999;14;0
999999;9999;rocket;57.878796;11.160669;6091.207544;121824.349;137;2;99;999;13;0
999999;9999;rocket;57.878696;11.160667;8098.097372;121471.6581;137;2;99;999;12;0

Now I would like to safe this dataframe with an additional header as a JSON file: The additional header looks like this dictionary:

headlines={
           "now": 1636008051.9,
           "messages": 6236,
           }

The aim JSON should contain the information given by "headlines" (but without its name) and the content of the dataframe:

{
  "now": 1636008051.9,
  "messages": 6236,
  "track": [
    {      
      "Param1": 999999,
      "Param2": "9999",
      "name": "rocket",
      "lat":  57.878696,
      "lon": 11.160667,
      "altitude": 1089,
      "vert_rate": 0,
      "track": 137,
      "speed": 2,
      "category": 99,
      "Param3": 999,
      "Param4": 16,
      "Param5": 0
    }
    {      
      "Param1": 999999,
      "Param2": "9999",
      "name": "rocket",
      "lat":  57.878796,
      "lon": 11.160668,
      "altitude": 2543.963336,
      "vert_rate": 152638.0483,
      "track": 137,
      "speed": 2,
      "category": 99,
      "Param3": 999,
      "Param4": 15,
      "Param5": 0
    }
    {      
      "Param1": 999999,
      "Param2": "9999",
      "name": "rocket",
      "lat":  57.878896,
      "lon": 11.160670,
      "altitude": 4226.050004,
      "vert_rate": 126781.7063,
      "track": 137,
      "speed": 2,
      "category": 99,
      "Param3": 999,
      "Param4": 14,
      "Param5": 0
    }
    {...and so on...}
  ]
}

The dataframe itself I can simply turn to JSON like that:

json = track.to_json(path_out + "result.json", orient='records')

but here I don't know how to add the preceding lines from the "header" dict How can I join the dictionary and the csv to output the JSON? Or is there a simpler way? Or any hint to a post which I didn't find? I need to do it in pandas as the csv-dataframe will be further needed.


Solution

  • Create a dictionary, assign it as a new dictionary key and export with json.dump:

    import json
    
    headlines['track'] = df.to_dict(orient='records')
    
    with open(path_out + 'result.json', 'w') as f:
        json.dump(headlines, f)
    

    Or as a string:

    import json
    
    headlines['track'] = df.to_dict(orient='records')
    
    print(json.dumps(headlines, indent=2))
    

    Output:

    {
      "now": 1636008051.9,
      "messages": 6236,
      "track": [
        {
          "Param1": 999999,
          "Param2": 9999,
          "name": "rocket",
          "lat": 57.878696,
          "lon": 11.160667,
          "altitude": 1089.0,
          "vert_rate": 0.0,
          "track": 137,
          "speed": 2,
          "category": 99,
          "Param3": 999,
          "Param4": 16,
          "Param5": 0
        },
        {
          "Param1": 999999,
          "Param2": 9999,
          "name": "rocket",
          "lat": 57.878796,
          "lon": 11.160668,
          "altitude": 2543.963336,
          "vert_rate": 152638.0483,
          "track": 137,
          "speed": 2,
          "category": 99,
          "Param3": 999,
          "Param4": 15,
          "Param5": 0
        },
        {
          "Param1": 999999,
          "Param2": 9999,
          "name": "rocket",
          "lat": 57.878896,
          "lon": 11.16067,
          "altitude": 4226.050004,
          "vert_rate": 126781.7063,
          "track": 137,
          "speed": 2,
          "category": 99,
          "Param3": 999,
          "Param4": 14,
          "Param5": 0
        },
        {
          "Param1": 999999,
          "Param2": 9999,
          "name": "rocket",
          "lat": 57.878796,
          "lon": 11.160669,
          "altitude": 6091.207544,
          "vert_rate": 121824.349,
          "track": 137,
          "speed": 2,
          "category": 99,
          "Param3": 999,
          "Param4": 13,
          "Param5": 0
        },
        {
          "Param1": 999999,
          "Param2": 9999,
          "name": "rocket",
          "lat": 57.878696,
          "lon": 11.160667,
          "altitude": 8098.097372,
          "vert_rate": 121471.6581,
          "track": 137,
          "speed": 2,
          "category": 99,
          "Param3": 999,
          "Param4": 12,
          "Param5": 0
        }
      ]
    }