python-3.xtensorflowkerasdeep-learningcaptions

AttributeError: 'str' object has no attribute 'ndim', unable to use model.predict()


I was trying to make an Image Captioning model in a similar fashion as in here

I used ResNet50 instead off VGG16 and also had to use progressive loading via model.fit_generator() method.

I used ResNet50 from here and when I imported it by setting include_top = False, It gave me features of photo in shape of {'key': [[[[value1, value2, .... value 2048]]]]}, where "key" is the image id.

Here's my code of caption generator function:-

def createCaptions(tokenizer, photoData, MaxLength, model):
    for key, feature in photoData.items():
        inSeq = "START"
        for i in range(MaxLength):
            sequence = tokenizer.texts_to_sequences([inSeq])[0]
            sequence = pad_sequences([sequence], maxlen = MaxLength)
            ID = model.predict([np.array(feature[0][0][0]), inSeq])
            ID = word_for_id(ID)
            if ID is None:
                break
            inSeq += " " + ID
            if ID == "END":
                break
        print(inSeq)

The function word_for_id is :-

def word_for_id(integer, tokenizer):
    for word, index in tokenizer.word_index.items():
        if index == integer:
            return word
    return None

I had generated photoData via:-

features = {}
for images in os.listdir(args["image"]):
    filename = args["image"] + '/' + images
    image = load_img(filename, target_size = inputShape)
    image = img_to_array(image)
    image = np.expand_dims(image, axis = 0)
    image = preprocess(image)
    pred = resnet.predict(image)
    image_id = images.split('.')[0]
    features[image_id] = pred
    print('>{}'.format(images))

features is my photoData dictionary.

When I try to generate captions:-

caption = createCaptions(tokenizerTrain, features, 34, model)

I get the following error:-

Traceback (most recent call last):
  File "CaptionGenerator.py", line 111, in <module>
caption = createCaptions(tokenizerTrain, features, 34, model)
  File "CaptionGenerator.py", line 101, in createCaptions
ID = model.predict([np.array(feature[0][0][0]), inSeq])
  File "/home/aditya/.virtualenvs/cv/lib/python3.5/site-packages/keras/engine/training.py", line 1817, in predict
check_batch_axis=False)
  File "/home/aditya/.virtualenvs/cv/lib/python3.5/site-packages/keras/engine/training.py", line 76, in _standardize_input_data
data = [np.expand_dims(x, 1) if x is not None and x.ndim == 1 else x for x in data]
  File "/home/aditya/.virtualenvs/cv/lib/python3.5/site-packages/keras/engine/training.py", line 76, in <listcomp>
data = [np.expand_dims(x, 1) if x is not None and x.ndim == 1 else x for x in data]
AttributeError: 'str' object has no attribute 'ndim'

Where did I go wrong? Please help. Thanks in advance.


Solution

  • You pass inSeq = "START" to model.predict as a string:

    ID = model.predict([np.array(feature[0][0][0]), inSeq])
    

    without pre-processing it. You will need to encode it to an array.