I'm trying to mask 8752 images with transformers like this
from transformers import pipeline
from PIL import Image
import requests
import cv2
import numpy as np
from matplotlib import pyplot as plt
semantic_segmentation_nvidia = pipeline("image-segmentation", "nvidia/segformer-b0-finetuned-ade-512-512")
jpeg_im = None
a = 0
mask_i = 0
f = open("masking_log.txt", "w")
for im in large_image_stack_512:
i=0
while(i == 0):
jpeg_im = Image.open(os.path.join(ROOT_DIR,im))
print(os.path.join(ROOT_DIR,im))
# Semantic segmentation
segmentation = semantic_segmentation_nvidia(jpeg_im)
print("the length of current segmentation labels are: ", len(segmentation))
water_mask_label = segmentation[mask_i]["label"]
print(water_mask_label)
print("here")
if (water_mask_label == "water"):
print("Successful labelling at: ", mask_i)
water_mask = segmentation[mask_i]["mask"]
print("here")
imar = np.asarray(water_mask)
print(water_mask_label)
print("type im (array)", type(imar))
f.write("image " + str(a) + "\nsuccess-label at " + str(mask_i) + "\nwith dir: " + str(im) + "\n with mask labeled as: " + str(water_mask_label) + '\n\n')
plt.imsave('D:\..\Data\'+'img_'+str(a)+'.jpg', imar, cmap="gray")
i=1
a+=1
mask_i= 0
semantic_jpeg = None
imar = None
water_mask = None
water_mask_label = None
segmentation = None
water_mask_label = None
else:
print("not water")
if (mask_i < len(segmentation)):
mask_i += 1
else:
f.write("image " + str(a) + "\n unsuccess-labelling (has no 'water' label)" + "final mask_i value: " + str(mask_i) + "\nwith dir: " + str(im) + "\n check later " + + '\n\n')
print("masking fails, check later image" + im)
i = 1
continue
#plt.imshow(water_mask)
#plt.show()
#print("type jpeg_im (jpeg)", type(water_mask))
continue
#print(len(cropped))
f.close()
Each of segmentation = semantic_segmentation_nvidia(jpeg_im) will have different size of array, for example with this image I have 11 items inside it like this:
The code (did this in jupyter notebook rows)
a_512 = semantic_segmentation_nvidia(image_512)
a_512
the output, a_512 variable is a List with 11 items inside it
[{'score': None,
'label': 'wall',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'building',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'sky',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'tree',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'earth',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'water',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'fence',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'railing',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'bridge',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'ship',
'mask': <PIL.Image.Image image mode=L size=512x512>},
{'score': None,
'label': 'pier',
'mask': <PIL.Image.Image image mode=L size=512x512>}]
To access things I need, probably the 'label', I need to access the PIL.Image.Imagedata from each List item like this
(a_512[5]["mask"]
In the codes above, I used a variable to represent 5 in this code because apparently each image has different order of when it detects 'water' as the label and thus the required mask.
Because python can compare strings, I'm trying to make a for loop to check every items in the list of every 8752 images. Let's say some of the images can have 10-12 items like this output of the final run.
D:\..\Data\cropped_512\img_2541.jpg
the length of current segmentation labels are: 15
car
here
not water
D:\..\Data\cropped_512\img_2541.jpg
the length of current segmentation labels are: 15
water
here
Successful labelling at: 7
here
water
type im (array) <class 'numpy.ndarray'>
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
wall
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
building
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
sky
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
tree
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
road
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
mountain
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
car
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
sea
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
fence
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
bridge
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
boat
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
ship
here
not water
D:\..\Data\cropped_512\img_2542.jpg
the length of current segmentation labels are: 12
---------------------------------------------------------------------------
IndexError Traceback (most recent call last)
Cell In[16], line 17
15 segmentation = semantic_segmentation_nvidia(jpeg_im)
16 print("the length of current segmentation labels are: ", len(segmentation))
---> 17 water_mask_label = segmentation[mask_i]["label"]
18 print(water_mask_label)
19 print("here")
IndexError: list index out of range
As you can see the index keeps increasing and didn't touch the last else of the if-else comparator, so it went out of bounds. I have tried this and using range on the for-loop but it didn't work and still went out of bounds. I put breaks and it stops the loop, I put continue it keeps went out of bounds.Which part did I do wrong and understood incorrectly about python comparator behavior?
I also tried logging it into .txt file or give status with print but it didn't go that way and always go out of bounds. Will gladly add more details if needed. The image is 512 x 512 size.
The problem is solved according to @OldBoy input to move the increment.
full code
mask_i = 0
a = 0
f = open("masking_log.txt", "w")
for im in large_image_stack_512:
image_success_flag = 0
mask_i= 0
while(image_success_flag < 1):
jpeg_im = Image.open(os.path.join(ROOT_DIR,im))
print(os.path.join(ROOT_DIR,im))
# Semantic segmentation
segmentation = semantic_segmentation_nvidia(jpeg_im)
print("the length of current segmentation labels are: ", len(segmentation))
while(mask_i < len(segmentation)):
image_mask = segmentation[mask_i]["label"]
print(image_mask)
if(image_mask == "water"):
print("correct mask")
water_mask = segmentation[mask_i]["mask"]
imar = np.asarray(water_mask)
plt.imsave('D:/semester_12/Data_Irredeano/'+'img_'+str(a)+'.jpg', imar, cmap="gray")
print("here")
f.write("image " + str(a) + "\nsuccess-label at " + str(mask_i) + "\nwith dir: " + str(im) + "\n with mask labeled as: " + str(water_mask_label) + '\n\n')
print("mask-succesfully saved")
mask_i = 0
break
elif(image_mask != "water"):
mask_i+=1
print("number of mask: ", mask_i)
if(mask_i == len(segmentation)):
print("this image has no correct mask, check later")
f.write("image " + str(a) + "\n unsuccess-labelling (has no 'water' label) final \n mask_i value: " + str(mask_i) + "\nwith dir: " + str(im) + "\n check later " + '\n\n')
image_success_flag=+1
a+=1
f.close()
Basically instead of selecting mask by checking the segmentation[mask_i]["label] I check if the 'cursor' or mask_i if it's smaller than the length of the List (len(segmentation)). The += also contribute to the problem since it's adding first then change the number, as implied here and here, because of that my 'cursor variable' can move beyond the array size before checking the segmentation[mask_i]["label]. But I don't think I have other choice other than that to increment since =+ is just redefining itself.
Other than that I also add another while condition to make sure the code runs while the mask_i is below the size of the List, so the program become "if it's still below the list size, check whether mask is "water" or not "water".
Although the program is finished and can mask most of the images, I still have to log several different images since not all of them had "water" as the label, but something close like "Sea" that we humans can intuitively say they are basically the same but computer that can only compare strings not.
Thank you again for everyone who's willing to help, I'm accepting if there's better way to do it