pythongesture-recognitionmediapipe

How to get value out of loop, without breaking the loop


I'm trying to make a Video Player, that uses gesture recognition for the video player actions (play, pause, fast-forward, etc.)

For the Video Player I use PyQt5, and for the gesture recognition I use MediaPipe. Here's what my program looks like

Now this is the code I use for running the Camera and emiting it in the QMainWindow:

class Camera(QThread):
    image_update = pyqtSignal(QImage)
    video_capture = cv2.VideoCapture(0)
    action = ""

    def run(self):
        self.active_thread = True
        video_capture = cv2.VideoCapture(0)

        tracker = htm.HandTrackingModule()
        recognize = grm.GestureRecognitionModule() 

        while self.active_thread:
            ret, frame = video_capture.read()
            h, w, c = frame.shape
            if ret:
                image = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
                landmarks = tracker.detect_hand(image, True)
                fingers = get_position(landmarks)

                action = recognize.do_gesture(position=fingers, landmarks=landmarks, img=image)

                convert_to_qt_format = QImage(image.data, image.shape[1], image.shape[0], QImage.Format_RGB888)

            pic = convert_to_qt_format.scaled(640, 480, Qt.KeepAspectRatio)

            self.image_update.emit(pic)

The do_gesture method of the GestureRecognitionModule, returns a string about what action should be done, according to the gesture recognition (pause, play, fast-forward, rewind, volume up/down, ...)

I want to get 'action' (do_gesture()'s value) in my QMainWindows so I can connect it with the VideoPlayer.

But I'm not sure how can I do that.

tldr: How can I get a value out from the loop, without breaking the loop?


Solution

  • You probably want to emit a signal.

    Add a signal for action to your Camera class, right alongside your image_update signal:

    class Camera(QThread):
        image_update = pyqtSignal(QImage)
        gesture_action = pyqtSignal(str)
    

    and then in your loop you can do:

                    action = recognize.do_gesture(position=fingers, landmarks=landmarks, img=image)
                    self.gesture_action.emit(action)
    

    This will not break the loop.

    Connect that signal to a slot (presumably in your MainWindow or one of its child widgets) that does whatever you want to do when there's an action.