pythoncomputer-visioncamerarosbasler

Problem with Basler Camera - Low Frame Rate and Latency - ROS


I have a Basler camera that is connected to my PC with a USB-3. When I visualize the output with RVIZ, it is very slow and the frame rate is low. For example, I pass in front of it at t=6 (second), and it shows me on t=9 (second). Also, the frame rate is so low although in the configuration file I determined the frame rate as 60.0.

I get this error on the terminal:

[ERROR] [1655490133.698416607]: Error: 3792703499 Not enough memory to submit transfer. You can increase the usbfs memory by modifying /sys/module/usbcore/parameters/usbfs_memory_mb.
[ERROR] [1655490133.698460259]: Error: Grab was not successful

Also, I get this warning on the terminal:

[ WARN] [1655491176.729972777]: Did not reach the target brightness before timeout of 5 sec! Stuck at brightness -1.77634e+08 and exposure 100000us

This is my configuration file:

#  The tf frame under which the images were published
camera_frame: pylon_camera

#  The DeviceUserID of the camera. If empty, the first camera found in the
#  device list will be used
device_user_id: "ros_basler_left"

#  The CameraInfo URL (Uniform Resource Locator) where the optional intrinsic
#  camera calibration parameters are stored. This URL string will be parsed
#  from the ROS-CameraInfoManager:
#  http://docs.ros.org/api/camera_info_manager/html/classcamera__info__manager_
#  1_1CameraInfoManager.html#details
camera_info_url: ""

#  The encoding of the pixels -- channel meaning, ordering, size
#  taken from the list of strings in include/sensor_msgs/image_encodings.h
#  The supported encodings are 'mono8', 'bgr8', 'rgb8', 'bayer_bggr8',
#  'bayer_gbrg8' and 'bayer_rggb8'
#  Default values are 'mono8' and 'rgb8'
image_encoding: "rgb8"

#  Binning factor to get downsampled images. It refers here to any camera
#  setting which combines rectangular neighborhoods of pixels into larger
#  "super-pixels." It reduces the resolution of the output image to
#  (width / binning_x) x (height / binning_y).
#  The default values binning_x = binning_y = 0 are considered the same
#  as binning_x = binning_y = 1 (no subsampling).
binning_x: 1
binning_y: 1

#  The desired publisher frame rate if listening to the topics.
#  This parameter can only be set once at startup
#  Calling the GrabImages-Action can result in a higher framerate
frame_rate: 60.0

#  Mode of camera's shutter.
#  The supported modes are "rolling", "global" and "global_reset"
#  Default value is "" (empty) means default_shutter_mode
#shutter_mode: ""

##########################################################################
######################## Image Intensity Settings ########################
##########################################################################
# The following settings do *NOT* have to be set. Each camera has default
# values which provide an automatic image adjustment resulting in valid
# images
##########################################################################

#  The exposure time in microseconds to be set after opening the camera.
exposure: 100000.0

#  The target gain in percent of the maximal value the camera supports
#  For USB-Cameras, the gain is in dB, for GigE-Cameras it is given in so
#  called 'device specific units'.
# gain: 0

#  Gamma correction of pixel intensity.
#  Adjusts the brightness of the pixel values output by the camera's sensor
#  to account for a non-linearity in the human perception of brightness or
#  of the display system (such as CRT).
# gamma: 1.0

#  The average intensity value of the images. It depends the exposure time
#  as well as the gain setting. If 'exposure' is provided, the interface will
#  try to reach the desired brightness by only varying the gain. (What may
#  often fail, because the range of possible exposure vaules is many
#  times higher than the gain range). If 'gain' is provided, the interface will
#  try to reach the desired brightness by only varying the exposure time. If
#  gain AND exposure are given, it is not possible to reach the brightness,
#  because both are assumed to be fix.
brightness: 100

#  Only relevant, if 'brightness' is set:
#  The brightness_continuous flag controls the auto brightness function.
#  If it is set to false, the brightness will only be reached once.
#  Hence changing light conditions lead to changing brightness values.
#  If it is set to true, the given brightness will be reached continuously,
#  trying to adapt to changing light conditions. This is only possible for
#  values in the possible auto range of the pylon API which is e.g. [50 - 205]
#  for acA2500-14um and acA1920-40gm
brightness_continuous: true

#  Only relevant, if 'brightness' is set:
#  If the camera should try to reach and / or keep the brightness, hence
#  adapting to changing light conditions, at least one of the following flags
#  must be set.
#  If both are set, the interface will use the profile that tries to keep the
#  gain at minimum to reduce white noise.
#  The exposure_auto flag indicates, that the desired brightness will be
#  reached by adapting the exposure time.
#  The gain_auto flag indicates, that the desired brightness will be
#  reached by adapting the gain.
exposure_auto: true
gain_auto: true

##########################################################################

#  The timeout while searching the exposure which is connected to the
#  desired brightness. For slow system this has to be increased.
# exposure_search_timeout: 5.0

#  The exposure search can be limited with an upper bound. This is to prevent
#  very high exposure times and resulting timeouts.
#  A typical value for this upper bound is ~2000000us.
# auto_exposure_upper_limit: 2000000.0

#  The MTU size. Only used for GigE cameras.
#  To prevent lost frames configure the camera has to be configured
#  with the MTU size the network card supports. A value greater 3000
#  should be good (1500 for RaspberryPI)
# gige:
#  mtu_size: 3000

#  Only used for GigE cameras.
#  The inter-package delay in ticks to prevent lost frames.
#  For most of GigE-Cameras, a value of 1000 is reasonable.
#  For cameras used on a RaspberryPI this value should be set to 11772.
# gige:
#  inter_pkg_delay: 1000


Solution

  • I solved it. There were several issues.

    1- You should install the 'Pylon Viewer' suite provided by Basler. During installation, it has some settings that configure your system properly. For example, large USB bandwidth, etc.

    2- RGB encoding is super slow, use Bayer RGB instead.

    3- Also, look at this link which disables a setting that is crucial: github issue. (it can be disabled via launch file or ...)

    4- Use 'pylon viewer' to find the setting in the GUI interface of the application, then save the setting into 'UserSet1'. Load the setting when you launch the camera node 5- This is my final config file:

    #  The tf frame under which the images were published
    camera_frame: pylon_camera
    
    #  The DeviceUserID of the camera. If empty, the first camera found in the
    #  device list will be used
    device_user_id: ""
    
    #  The CameraInfo URL (Uniform Resource Locator) where the optional intrinsic
    #  camera calibration parameters are stored. This URL string will be parsed
    #  from the ROS-CameraInfoManager:
    #  http://docs.ros.org/api/camera_info_manager/html/classcamera__info__manager_
    #  1_1CameraInfoManager.html#details
    camera_info_url: ""
    
    #  The encoding of the pixels -- channel meaning, ordering, size
    #  taken from the list of strings in include/sensor_msgs/image_encodings.h
    #  The supported encodings are 'mono8', 'bgr8', 'rgb8', 'bayer_bggr8',
    #  'bayer_gbrg8' and 'bayer_rggb8'
    #  Default values are 'mono8' and 'rgb8'
    image_encoding: "bayer_rggb8"
    
    #  Binning factor to get downsampled images. It refers here to any camera
    #  setting which combines rectangular neighborhoods of pixels into larger
    #  "super-pixels." It reduces the resolution of the output image to
    #  (width / binning_x) x (height / binning_y).
    #  The default values binning_x = binning_y = 0 are considered the same
    #  as binning_x = binning_y = 1 (no subsampling).
    # binning_x: 1
    # binning_y: 1
    
    #  The desired publisher frame rate if listening to the topics.
    #  This parameter can only be set once at startup
    #  Calling the GrabImages-Action can result in a higher framerate
    frame_rate: 35.0
    
    trigger_timeout: 500  # in ms
    grab_timeout: 500  # in ms
    
    # Camera grab strategy 
    # 0 : GrabStrategy_OneByOne
    # 1 : GrabStrategy_LatestImageOnly
    # 2 : GrabStrategy_LatestImages
    grab_strategy : 2
    
    ##--White Balance Auto--##
    # Automatically corrects color shifts in images acquired. 0 = Off, 1 = Once, 2 = Continuous
    white_balance_auto : 2
    ##----------------------##
    
    ##--White balance--##
    # The Balance White camera feature allows you to manually correct color shifts so that white objects appear white in images acquired.
    # The increase or decrease in intensity is proportional. For example, if the balance ratio for a color is set to 1.2, the intensity of that color is increased by 20 %.
    # Activating these values will turn off the white_balance_auto.
    # white_balance_ratio_red : 1.0
    # white_balance_ratio_green : 1.0
    # white_balance_ratio_blue : 1.0
    ##--------------##
    
    #  Mode of camera's shutter.
    #  The supported modes are "rolling", "global" and "global_reset"
    #  Default value is "" (empty) means default_shutter_mode
    #shutter_mode: ""
    
    ##########################################################################
    ######################## Image Intensity Settings ########################
    ##########################################################################
    # The following settings do *NOT* have to be set. Each camera has default
    # values which provide an automatic image adjustment resulting in valid
    # images
    ##########################################################################
    
    #  The exposure time in microseconds to be set after opening the camera.
    # exposure: 10000.0
    
    #  The target gain in percent of the maximal value the camera supports
    #  For USB-Cameras, the gain is in dB, for GigE-Cameras it is given in so
    #  called 'device specific units'.
    # gain: 0.5
    
    #  Gamma correction of pixel intensity.
    #  Adjusts the brightness of the pixel values output by the camera's sensor
    #  to account for a non-linearity in the human perception of brightness or
    #  of the display system (such as CRT).
    # gamma: 1.0
    
    #  The average intensity value of the images. It depends the exposure time
    #  as well as the gain setting. If 'exposure' is provided, the interface will
    #  try to reach the desired brightness by only varying the gain. (What may
    #  often fail, because the range of possible exposure vaules is many
    #  times higher than the gain range). If 'gain' is provided, the interface will
    #  try to reach the desired brightness by only varying the exposure time. If
    #  gain AND exposure are given, it is not possible to reach the brightness,
    #  because both are assumed to be fix.
    brightness: 100
    
    #  Only relevant, if 'brightness' is set:
    #  The brightness_continuous flag controls the auto brightness function.
    #  If it is set to false, the brightness will only be reached once.
    #  Hence changing light conditions lead to changing brightness values.
    #  If it is set to true, the given brightness will be reached continuously,
    #  trying to adapt to changing light conditions. This is only possible for
    #  values in the possible auto range of the pylon API which is e.g. [50 - 205]
    #  for acA2500-14um and acA1920-40gm
    brightness_continuous: true
    
    #  Only relevant, if 'brightness' is set:
    #  If the camera should try to reach and / or keep the brightness, hence
    #  adapting to changing light conditions, at least one of the following flags
    #  must be set.
    #  If both are set, the interface will use the profile that tries to keep the
    #  gain at minimum to reduce white noise.
    #  The exposure_auto flag indicates, that the desired brightness will be
    #  reached by adapting the exposure time.
    #  The gain_auto flag indicates, that the desired brightness will be
    #  reached by adapting the gain.
    exposure_auto: true
    gain_auto: true
    
    ##########################################################################
    
    #  The timeout while searching the exposure which is connected to the
    #  desired brightness. For slow system this has to be increased.
    # exposure_search_timeout: 5.0
    
    #  The exposure search can be limited with an upper bound. This is to prevent
    #  very high exposure times and resulting timeouts.
    #  A typical value for this upper bound is ~2000000us.
    # auto_exposure_upper_limit: 2000000.0
    
    #  The MTU size. Only used for GigE cameras.
    #  To prevent lost frames configure the camera has to be configured
    #  with the MTU size the network card supports. A value greater 3000
    #  should be good (1500 for RaspberryPI)
    # gige:
    #  mtu_size: 3000
    
    #  Only used for GigE cameras.
    #  The inter-package delay in ticks to prevent lost frames.
    #  For most of GigE-Cameras, a value of 1000 is reasonable.
    #  For cameras used on a RaspberryPI this value should be set to 11772.
    # gige:
    #  inter_pkg_delay: 1000