I am trying to assess whether the lips of a person are moving too much while the mouth is closed (to conclude it is chewing).
The mouth closed part is done without any issue, but when I try to assess the lip movement through landmarks (dlib) there seems to be a problem with the last landmark of the mouth.
Inspired by the mouth example (https://github.com/mauckc/mouth-open/blob/master/detect_open_mouth.py#L17), I wrote the following function:
def lips_aspect_ratio(shape):
# grab the indexes of the facial landmarks for the lip
(mStart, mEnd) = (61, 68)
lip = shape[mStart:mEnd]
print(len(lip))
# compute the euclidean distances between the two sets of
# vertical lip landmarks (x, y)-coordinates
# to reach landmark 68 I need to get lib[7] not lip[6] (while I get lip[7] I get IndexOutOfBoundError)
A = dist.euclidean(lip[1], lip[6]) # 62, 68
B = dist.euclidean(lip[3], lip[5]) # 64, 66
# compute the euclidean distance between the horizontal
# lip landmark (x, y)-coordinates
C = dist.euclidean(lip[0], lip[4]) # 61, 65
# compute the lip aspect ratio
mar = (A + B) / (2.0 * C)
# return the lip aspect ratio
return mar
The landmark of the lips are (61, 68), when I extract the lip as lip = shape[61:68] and try to access the last landmark as lip[7] I get the following error:
IndexError: index 7 is out of bounds for axis 0 with size 7
Why is that? and How to get the last landmark of the lip/face
lip = shape[61:68]
The slices exclude the end element. So you got 7 elements: 61,62,63,64, 65,66,67. And len(lip) == 7 confirms that.
If there truly are 8 points for the lip shape, and they include element 68, the slice should be:
lip = shape[61:69]
assert(len(range(61, 69) == 8)
assert(len(lip) == 8)
Note that if shape is sufficiently long, then len(shape[a:b:c]) == len(range(a, b, c)). That's because the range function acts like taking a slice out of an infinitely long list of integers (with caveats).
So, the problem is a classic off-by-one, not much to do with AI/image analysis :)