pythonstoragestoring-datastoring-information

How do I store my results which are of the form of graphs and numerical outputs/ Is there a universal way of storing any kind of result in python?


I am writing a program that runs for a long time. I want to run this program many times so that I can see the dependence of my results on the tweaking of parameters. So, suppose a situation similar to the following:

parameter=1

"Big code that takes a long time"


print(output, "output that depends on t")
plt.plot(x,y)

Now change the parameter to 2 and re-run again. I want to be able to pull the results of the previous one so that I can compare them.

So I want to sort of store them somehow so that the next time I need to look at the results I just have to execute a few lines and the stored results come up really quickly.


Solution

  • You can store all the information such as the inputs, params, and outputs in a dictionary. You can then use the dict to do further plotting and analysis.

    Here I add a minimal reproducible example. You can use this as a reference for your needs. The below code produces this plot as an output.input_vs_output_various_params

    import matplotlib.pyplot as plt
    import numpy as np
    import random
    
    def big_code(param, input):
        output = [i + param**(random.randrange(2, 5)) for i in input]
        return output
    
    def plot_experiments(info):
        rows, cols = 1, 6
        _, axs = plt.subplots(rows,cols)
        i = 0
    
        for val in info.values():
            param_idx = val['param']
            axs[i].plot(val['input'], val['output'])
            axs[i].set_title(f'param {param_idx}')       
            i+=1
    
        for ax in axs.flat:
            ax.set(xlabel='x-label', ylabel='y-label')
    
        # Hide x labels and tick labels for top plots and y ticks for right plots.
        for ax in axs.flat:
            ax.label_outer()
        plt.show()
    
    if __name__ == '__main__':
        input_params = [1,2,3,4,5,6]
        input_list = np.array(list(range(2000)))
        info = {}
        
        for exp_id ,param in enumerate(input_params):
            # Run your big code to get output
            output = big_code(param, input_list)
    
            # Save your output to a dataframe
            info[exp_id] = {'input': input_list, 'output': output, 'param': param }
            
        # Access your dict and plot
        plot_experiments(info)