I am currently developing a robot that uses YOLOv8 for real-time object detection based on the video captured by the robot’s camera. However, when the robot rotates, the input image becomes blurred, which causes a significant drop in detection accuracy. As a result, the robot has to either slow down or pause every 10 degrees during rotation, leading to poor efficiency.
To address this issue, I attempted to apply Sobel edge detection. Since the robot’s change in direction involves only rotation, which corresponds to horizontal movement (along the x-axis) in the camera view, I used a Sobel filter that emphasizes vertical edges by filtering x-axis signals. I then fed the processed images into a YOLOv8 model that had been trained on a dataset pre-processed with the same Sobel filtering.
However, despite this approach, the robot still fails to accurately detect objects while rotating.
while(True):
ret, frame = cap.read()
if not ret:
print("Can't receive frame (stream end?). Exiting ...")
break
gray = cv2.cvtColor(frame, cv2.COLOR_BGR2GRAY)
sobel_x = cv2.Sobel(gray, cv2.CV_64F, 1, 0, ksize = 3)
#sobel_y = cv2.Sobel(gray, cv2.CV_64F, 0, 1, ksize = 3)
#sobel = cv2.magnitude(sobel_x, sobel_y)
#sobel = np.uint8(np.clip(sobel, 0, 255))
absX = cv.convertScaleAbs(sobel_x)
absX_rgb = cv2.cvtColor(absX, cv2.COLOR_GRAY2BGR)
cv2.imshow("Sobel Edge3", absX)
results = trt_model(absX_rgb)
for result in results:
boxes = result.boxes # Boxes object for bounding box outputs
center = boxes.xyxy
shuttlecock_prob = boxes.conf
##########other process
if cv2.waitKey(1) == ord('q'):
break
This is the result when the robot is static Detection result when the robot is static
The result with sobel filter(I displayed the results in RGB colors for easier observation.) when the robot is turning Sobel filter result
The result with blurred dataset when the robot is turning Blurred dataset result
Another attempt with this approach was to annotate the blurry images captured by the robot and include them in the training dataset. Although this helped improve object recognition under blurry conditions, it resulted in a decrease in overall accuracy.
That is motion blur.
Do not try to "filter it away". That will not work.
Motion blur is reduced using more light and shorter exposure time. Or strobe light synchronized to image exposures, which you cannot do with an arbitrary webcam.
