How to write Python dictionaries to file without make them strings?

I would like to write a list of Python dictionaries into a file. However, I need the dictionaries (and lists within) to remain dictionaries, i.e. when I load the file for processing I want them to use dictionaries and not have to work with strings.

Here is my sample code which write the data as strings, is there a way to retain the origin Python data structures (in real code the list data has hundreds of dictionaries, each of which may have hundreds of lists as values). I cannot simple pickle the data, for a number of reasons (one of which is the file needs to be human readable).

import csv
import pandas as pd

def write_csv_file(data, iteration):
    with open('%s.csv' % 'name', 'wb') as data_csv:
        writer_data = csv.writer(data_csv, delimiter=',')
        for d in data:
            writer_data.writerow([iteration] + [d])


data = [{'a':1, 'b':2}, {'e':[1], 'f':[2,10]}]
iteration = 1
write_csv_file(data, iteration)

At the moment I read the data file using pandas in the following manner to process the data.

d = pd.read_csv('name.csv')
d = pd.DataFrame(d)

Solution

Starting with Python 2.6 you can use ast.literal_eval.

import ast
ast.literal_eval('{"a":1, "b":2, "c":3}')
{'a': 1, 'b': 2, 'c': 3}

If whole column of your pandas DataFrame is a dict, then you can save it to CSV normaly (with separator other than ,) and then map this dictionary-like column strings into dictionaries:

df['DICTIONARY_COLUMN'].map(ast.literal_eval)

Naturally you can ignore dataframe part - you can transform them also in loop or whatever way you like. Important parts are: ast.literal_eval and using non-comma separator (because you use commas in dict-like strings).