pythonpandasnumpycsvsplit

Split columns in Csv file


I have a CSV file and it's pretty messy. The first column is fine, but all the rest of the data is in the second column. All data as VariableName1=Variable1, VariableName2=Variable2, VariableName3=Variable3, ... are in the second column.

<div class="jp-RenderedText jp-OutputArea-output" data-mime-type="text/plain">
<pre>              var1                                            var2  \
1    SfgvbdvbUJ05-1  var3=10,var4=/a/n/anghelo_rujo_edited-...   
2      OLBCANGR15  var3=10,var4=/c/a/cangrande_test.jpg,a...   
3        ZAMdvFIA19  var3=10,var4=/p/i/pierluigi_zampaglion...   
4        VINMUL18  var3=10,var4=/r/u/rudi_vindimian_mulle...   
5        PRACLA16  var3=10,var4=/p/r/pracla16_podere_prad...   
..            ...                                                ...   
175        WALLIM  var3=25,var4=/w/a/walcher_limoncello_w...   
239       SMROS20  var3=10,var4=/s/e/sella_e_mosca_rosato...   
288     SAELAMB19  var3=10,var6=Modena,bottleml=750,box_size=1...   
343        DILABB  var3=40,var4=/d/i/dilabb_distillerie_l...   
357       VANER19  var3=10,var4=/v/a/valdibella_kerasos_v...   

     var4  ...  var9  var10  var11  
1          NaN  ...   NaN           NaN            NaN  
2          NaN  ...   NaN           NaN            NaN  
3          NaN  ...   NaN           NaN            NaN  
4          NaN  ...   NaN           NaN            NaN  
5          NaN  ...   NaN           NaN            NaN  
..         ...  ...   ...           ...            ...  
175        NaN  ...   NaN           NaN            NaN  
239        NaN  ...   NaN           NaN            NaN  
288        NaN  ...   NaN           NaN            NaN  
343        NaN  ...   NaN           NaN            NaN  
357        NaN  ...   NaN           NaN            NaN  


</pre>
</div>

I took the second column as separate new data and split it with ,. But I can't separate the VariableName1=Variable1 data into VariableName columns.

When I do it with String Contains, I get stuck on the =... part.

Please, help me. I'm in trouble with this CSV. What I want is to have that value under each column name.

var1          var2         var3         var4
ZAMffFIA19     10           2         /a/n/anghelo_rujo_edited...
VINMUfgvL18    25           1         /r/u/rudi_vindimian_mulle...

Solution

  • After much effort, I developed an algorithm that solved my problem. Here is the code:

    def parse_csv_file(input_file_name,output_file_name):
        with open(input_file_name) as file:
            lines = file.readlines()
            keys=lines[0]
            num_rows=len(lines)
            lines=lines[1:num_rows]
            sku_lst=[]
            for line in lines:
                line=line.replace('"','')
                line=line.replace('\n','')
                line_splt=line.split(',')
                sku_dict={}
                sku_dict['sku']=line_splt[0]
                for elm in line_splt[1:-1]:
                    if(elm!=''):
                        elm_splt=elm.split('=')
                        try:
                            sku_dict[elm_splt[0]]=elm_splt[1]
                        except Exception as e:
                            print(elm,'\n')
                            print(line)
                            print(elm_splt,'\n')
                            print(e,'\n\n\n')
                sku_lst.append(sku_dict)
                output=pd.DataFrame(sku_lst).fillna('')
                output.to_csv(output_file_name)
        return(output)
    

    Then let's run this function.

    df=parse_csv_file('data_mixed.csv','data_cleaned.csv')
    

    If you want to access the processes and code development stages of this problem >> github repo link