pythonpandasdataframecsvmri

3D array saved as data frame in csv is not successful


I have some MRI data and I am trying to put the processed 3D images as arrays with corresponding labels to a data frame. Then save it in CSV file. I made each of the 3D images to a 161616 array.

However, after saving the CSV file, the arrays did not seem to be completely written in the file. Only part of them being saved.

I am not sure about why...

Here are my codes to process the 3D images:

trainDWI_images=[]
trainDWI_data=[]
for ptname in Train_uniqueID: 
  if Train_df[(Train_df.ProxID==ptname)&(Train_df.Name =='ep2d_diff_tra_DYNDIST0')].index.tolist() !=[]:
    index=Train_df[(Train_df.ProxID==ptname)&(Train_df.Name =='ep2d_diff_tra_DYNDIST0')].index.tolist()
  elif Train_df[(Train_df.ProxID==ptname)&(Train_df.Name == 'ep2d_diff_tra_DYNDIST_MIX0')].index.tolist() !=[]:
    index=Train_df[(Train_df.ProxID==ptname)&(Train_df.Name =='ep2d_diff_tra_DYNDIST_MIX0')].index.tolist()
  

  ijk_value=Train_df.ijk[index] #lesion location
  fid_value=Train_df.fid[index].values #label
  for i, ijk in enumerate(ijk_value.values):
    seed=[int(x) for x in ijk.split()]
    finding=fid_value[i]
    for filename in Train_total_DWI:
      if ptname in filename:
        i=Train_total_DWI.index(filename)
        niipath=os.path.join(train_dir,filename)   
        img_nii=nib.load(niipath)
        fourdimage=np.asanyarray(img_nii.dataobj)
        image = fourdimage[:, :, :, 0]
        img_arr=standardize(image)
        extracted_traint2images=img_arr[(seed[0]-8):(seed[0]+8),(seed[1]-8):(seed[1]+8),(seed[2]-8):(seed[2]+8)]#expanding the sub volume from lesion
        #print(extracted_traint2images.shape)
     
        trainDWI_images.append(extracted_traint2images)
        trainDWI_data.append({'Patient' : ptname , 'label' :finding,'images':extracted_traint2images , 'ijk':seed })
        #T2_df=T2_dframe.({'Patient' : ptname , 'label' :fid_value,'images':extracted_t2images , 'ijk':seed } , ignore_index=True)

trainDWI_df = pd.DataFrame(trainDWI_data)

Here are the codes I save as the CSV file:

outtrdwi = 'Packed_DWI_train.csv'
outdir = './dir'
if not os.path.exists(outdir):
    os.mkdir(outdir)

fullname = os.path.join(outdir, outtrdwi)    

trainDWI_df.to_csv(fullname, encoding='utf-8')

I noticed that in my CSV file, some part of the arrays are not saved and being replaced by '...'

Like this :

 [-0.32482308 -0.2481549  -0.2481549  ... -0.2481549  -0.13315264
   -0.47815944]

How can I fix this? Thank you.


Solution

  • I am trying to better understand the problem you are having. Looking at the lines:

     for i, ijk in enumerate(ijk_value.values):
        seed=[int(x) for x in ijk.split()]
        finding=fid_value[i]
    

    is [-0.32482308 -0.2481549 -0.2481549 ... -0.2481549 -0.13315264 -0.47815944] an example of the 'seed' variable from above? If so I assume that ijk is a string representation of those numbers in your dataset separated by a space. Before I continue, are my assumptions accurate?