I have some MRI data and I am trying to put the processed 3D images as arrays with corresponding labels to a data frame. Then save it in CSV file. I made each of the 3D images to a 161616 array.
However, after saving the CSV file, the arrays did not seem to be completely written in the file. Only part of them being saved.
I am not sure about why...
Here are my codes to process the 3D images:
trainDWI_images=[]
trainDWI_data=[]
for ptname in Train_uniqueID:
if Train_df[(Train_df.ProxID==ptname)&(Train_df.Name =='ep2d_diff_tra_DYNDIST0')].index.tolist() !=[]:
index=Train_df[(Train_df.ProxID==ptname)&(Train_df.Name =='ep2d_diff_tra_DYNDIST0')].index.tolist()
elif Train_df[(Train_df.ProxID==ptname)&(Train_df.Name == 'ep2d_diff_tra_DYNDIST_MIX0')].index.tolist() !=[]:
index=Train_df[(Train_df.ProxID==ptname)&(Train_df.Name =='ep2d_diff_tra_DYNDIST_MIX0')].index.tolist()
ijk_value=Train_df.ijk[index] #lesion location
fid_value=Train_df.fid[index].values #label
for i, ijk in enumerate(ijk_value.values):
seed=[int(x) for x in ijk.split()]
finding=fid_value[i]
for filename in Train_total_DWI:
if ptname in filename:
i=Train_total_DWI.index(filename)
niipath=os.path.join(train_dir,filename)
img_nii=nib.load(niipath)
fourdimage=np.asanyarray(img_nii.dataobj)
image = fourdimage[:, :, :, 0]
img_arr=standardize(image)
extracted_traint2images=img_arr[(seed[0]-8):(seed[0]+8),(seed[1]-8):(seed[1]+8),(seed[2]-8):(seed[2]+8)]#expanding the sub volume from lesion
#print(extracted_traint2images.shape)
trainDWI_images.append(extracted_traint2images)
trainDWI_data.append({'Patient' : ptname , 'label' :finding,'images':extracted_traint2images , 'ijk':seed })
#T2_df=T2_dframe.({'Patient' : ptname , 'label' :fid_value,'images':extracted_t2images , 'ijk':seed } , ignore_index=True)
trainDWI_df = pd.DataFrame(trainDWI_data)
Here are the codes I save as the CSV file:
outtrdwi = 'Packed_DWI_train.csv'
outdir = './dir'
if not os.path.exists(outdir):
os.mkdir(outdir)
fullname = os.path.join(outdir, outtrdwi)
trainDWI_df.to_csv(fullname, encoding='utf-8')
I noticed that in my CSV file, some part of the arrays are not saved and being replaced by '...'
Like this :
[-0.32482308 -0.2481549 -0.2481549 ... -0.2481549 -0.13315264
-0.47815944]
How can I fix this? Thank you.
I am trying to better understand the problem you are having. Looking at the lines:
for i, ijk in enumerate(ijk_value.values):
seed=[int(x) for x in ijk.split()]
finding=fid_value[i]
is [-0.32482308 -0.2481549 -0.2481549 ... -0.2481549 -0.13315264 -0.47815944]
an example of the 'seed' variable from above?
If so I assume that ijk is a string representation of those numbers in your dataset separated by a space. Before I continue, are my assumptions accurate?