I've got a live stream of JPG images coming in on a different thread in my Java app and I want to continually scan for faces to output later a list of all the different faces that passed the camera while it was running, and how many times each face was seen. Here's my current code:
void doImageProcessing() {
// Create face stuff
FKEFaceDetector faceDetector = new FKEFaceDetector(new HaarCascadeDetector());
EigenFaceRecogniser<KEDetectedFace, Person> faceRecognizer = EigenFaceRecogniser.create(512, new RotateScaleAligner(), 512, DoubleFVComparison.CORRELATION, Float.MAX_VALUE);
FaceRecognitionEngine<KEDetectedFace, Extractor<KEDetectedFace>, Person> faceEngine = FaceRecognitionEngine.create(faceDetector, faceRecognizer);
// Start loop
while (true) {
// Get next frame
byte[] imgData = nextProcessingData;
nextProcessingData = null;
// Decode image
BufferedImage img = ImageIO.read(new ByteArrayInputStream(imgData));
// Detect faces
FImage fimg = ImageUtilities.createFImage(img);
List<KEDetectedFace> faces = faceEngine.getDetector().detectFaces(fimg);
// Go through detected faces
for (KEDetectedFace face : faces) {
// Find existing person for this face
Person person = null;
try {
List<IndependentPair<KEDetectedFace, ScoredAnnotation<Person>>> rfaces = faceEngine.recogniseBest(face.getFacePatch());
ScoredAnnotation<Person> score = rfaces.get(0).getSecondObject();
if (score != null)
person = score.annotation;
} catch (Exception e) {
}
// If not found, create
if (person == null) {
// Create person
person = new Person();
System.out.println("Identified new person: " + person.getIdentifier());
// Train engine to recognize this new person
faceEngine.train(person, face.getFacePatch());
} else {
// This person has been detected before
System.out.println("Identified existing person: " + person.getIdentifier());
}
}
}
}
The problem is it always detects a face as new, even if it's the same face that was detected in the previous frame. rfaces is always empty. It can never identify an existing face. What am I doing wrong?
Also, I have no idea what the parameters for the EigenFaceRecognizer creator function should be, maybe that's why it's not recognizing anything...
The parameters you've given to the EigenFaceRecogniser.create() function are way off, so that's probably the likely cause of your problems. The following is more likely to work:
EigenFaceRecogniser<KEDetectedFace, Person> faceRecognizer = EigenFaceRecogniser.create(20, new RotateScaleAligner(), 1, DoubleFVComparison.CORRELATION, 0.9f);
Explaination:
The first parameter is the number of principal components in the EigenFace algorithm; the exact value is normally determined experimentally, but something around ~20 is probably fine.
The third parameter is the number of nearest neighbours to use for the KNN classifier. 1 nearest-neighbour should be fine.
The final parameter is a distance threshold for the classifier. The correlation comparison returns a similarity measure (high values mean more similar), so the threshold given is a lower limit that must be exceeded. As we've set 1 nearest neighbour, then the distance between the most similar face and the query face must be greater than 0.9. Note that a value of 0.9 is just a guess; in order to optimise the performance of your recogniser you'll need to play around with this.
Another minor point - instead of:
BufferedImage img = ImageIO.read(new ByteArrayInputStream(imgData));
FImage fimg = ImageUtilities.createFImage(img);
It's generally better to let OpenIMAJ read your image as it works around a number of known problems with ImageIO's handling of certain types of JPEG:
FImage fimg = ImageUtilities.readF(new ByteArrayInputStream(imgData));