gstreamerqualcommlinaro

Gstreamer1.0 missing element : v4l2h264enc in Qualcomm RB5


Before starting I am aware there are almost same questions with what I will ask, but since the question is almost board specific I will still ask.

I have a Qualcomm RB5 Robotic Vision Kit which has ov9282 camera. I configured a lighter Ubuntu 18.04, and kernel version is 5.15.0(5.15.0-qcomlt-arm64). I have built the gstreamer1.0(1.20.2) from source code. The meson command I used:

meson configure build -Dgst-plugins-bad:v4l2codecs=enabled -Dgst-plugins-good:v4l2=enabled -Dgst-plugins-good:v4l2-gudev=enabled

As you can see, I enabled all v4l2 related plugins and elements, and it seems I was successful partially. After meson, I executed ninja install to create symlinks. So far I haven't encountered with a problem. when I prompt gst-inspect-1.0 video4linux2, it shows elements in the .so file.

root@linaro-developer:~# gst-inspect-1.0 video4linux2
Plugin Details:
  Name                     video4linux2
  Description              elements for Video 4 Linux
  Filename                 /usr/local/lib/aarch64-linux-gnu/gstreamer-1.0/libgstvideo4linux2.so
  Version                  1.20.2.1
  License                  LGPL
  Source module            gst-plugins-good
  Binary package           GStreamer Good Plug-ins git
  Origin URL               Unknown package origin

  v4l2deviceprovider: Video (video4linux2) Device Provider
  v4l2h264dec: V4L2 H264 Decoder
  v4l2h265dec: V4L2 H265 Decoder
  v4l2mpeg2dec: V4L2 MPEG2 Decoder
  v4l2radio: Radio (video4linux2) Tuner
  v4l2sink: Video (video4linux2) Sink
  v4l2src: Video (video4linux2) Source
  v4l2vp8dec: V4L2 VP8 Decoder
  v4l2vp9dec: V4L2 VP9 Decoder

  9 features:
  +-- 8 elements
  +-- 1 device providers

However, in the build directory I can see gstv4l2h264enc.c.o file which I need.

root@linaro-developer:/home/linaro/gstreamer/build/subprojects/gst-plugins-good/sys/v4l2/libgstvideo4linux2.so.p# ls
gstv4l2.c.o        gstv4l2colorbalance.c.o    gstv4l2h263enc.c.o    gstv4l2h265enc.c.o     gstv4l2mpeg4enc.c.o  gstv4l2src.c.o    gstv4l2videoenc.c.o   gstv4l2vp9codec.c.o  tunernorm.c.o
gstv4l2allocator.c.o   gstv4l2deviceprovider.c.o  gstv4l2h264codec.c.o  gstv4l2jpegenc.c.o     gstv4l2object.c.o    gstv4l2transform.c.o  gstv4l2vidorient.c.o  gstv4l2vp9enc.c.o    v4l2-utils.c.o
gstv4l2bufferpool.c.o  gstv4l2element.c.o     gstv4l2h264enc.c.o    gstv4l2mpeg2codec.c.o  gstv4l2radio.c.o     gstv4l2tuner.c.o      gstv4l2vp8codec.c.o   tuner.c.o        v4l2_calls.c.o
gstv4l2codec.c.o       gstv4l2fwhtenc.c.o     gstv4l2h265codec.c.o  gstv4l2mpeg4codec.c.o  gstv4l2sink.c.o      gstv4l2videodec.c.o   gstv4l2vp8enc.c.o tunerchannel.c.o

Moreover, I viewed the content of the .so file to be sure I built the gstv4l2 module correctly. Output of nm -D libgstvideo4linux2.so, I can see the encoding parameters in the file.

U GST_CAT_DEFAULT
                 w _ITM_deregisterTMCloneTable
                 w _ITM_registerTMCloneTable
                 w __cxa_finalize@GLIBC_2.17
                 U __errno_location@GLIBC_2.17
                 U __getauxval@GLIBC_2.17
                 w __gmon_start__
                 U _gst_caps_features_memory_system_memory
                 U _gst_caps_type
                 U _gst_debug_category_new
                 U _gst_debug_get_category
                 U _gst_debug_min
                 U _gst_debug_register_funcptr
                 U _gst_element_error_printf
                 U _gst_fraction_range_type
                 U _gst_fraction_type
                 U _gst_int_range_type
                 U _gst_structure_type
                 U _gst_value_list_type
                 U clock_gettime@GLIBC_2.17
                 U close@GLIBC_2.17
                 U dup@GLIBC_2.17
                 U g_ascii_strcasecmp
                 U g_ascii_strdown
                 U g_ascii_strtod
                 U g_ascii_table
                 U g_ascii_tolower
                 U g_assertion_message_expr
                 U g_clear_error
                 U g_cond_broadcast
                 U g_cond_clear
                 U g_cond_init
                 U g_cond_signal
                 U g_cond_wait
                 U g_datalist_clear
                 U g_datalist_id_get_data
                 U g_datalist_id_set_data_full
                 U g_enum_register_static
                 U g_error_new_literal
                 U g_flags_register_static
                 U g_free
                 U g_get_real_time
                 U g_getenv
                 U g_idle_source_new
                 U g_intern_static_string
                 U g_list_delete_link
                 U g_list_find_custom
                 U g_list_foreach
                 U g_list_free
                 U g_list_free_full
                 U g_list_insert_sorted_with_data
                 U g_list_prepend
                 U g_list_remove
                 U g_list_reverse
                 U g_list_sort
                 U g_log
                 U g_main_context_new
                 U g_main_context_push_thread_default
                 U g_main_context_ref
                 U g_main_context_unref
                 U g_main_loop_new
                 U g_main_loop_quit
                 U g_main_loop_ref
                 U g_main_loop_run
                 U g_main_loop_unref
                 U g_malloc
                 U g_malloc0
                 U g_mutex_lock
                 U g_mutex_unlock
                 U g_object_class_install_property
                 U g_object_get
                 U g_object_new
                 U g_object_ref
                 U g_object_ref_sink
                 U g_object_set
                 U g_object_unref
                 U g_once_init_enter
                 U g_once_init_leave
                 U g_param_spec_boolean
                 U g_param_spec_boxed
                 U g_param_spec_enum
                 U g_param_spec_flags
                 U g_param_spec_int
                 U g_param_spec_string
                 U g_param_spec_uint
                 U g_path_get_basename
                 U g_quark_from_string
                 U g_quark_to_string
                 U g_queue_clear
                 U g_queue_foreach
                 U g_queue_push_tail
                 U g_rec_mutex_lock
                 U g_rec_mutex_unlock
                 U g_return_if_fail_warning
                 U g_set_error_literal
                 U g_signal_connect_data
                 U g_signal_emit
                 U g_signal_emit_by_name
                 U g_signal_handler_block
                 U g_signal_handler_disconnect
                 U g_signal_handler_unblock
                 U g_signal_new
                 U g_slice_alloc
                 U g_slice_free1
                 U g_slist_foreach
                 U g_slist_free
                 U g_slist_insert_sorted
                 U g_source_attach
                 U g_source_set_callback
                 U g_source_unref
                 U g_str_equal
                 U g_strcmp0
                 U g_strconcat
                 U g_strdup
                 U g_strdup_printf
                 U g_strerror
                 U g_thread_join
                 U g_thread_new
                 U g_type_add_interface_static
                 U g_type_check_instance_is_a
                 U g_type_check_value
                 U g_type_check_value_holds
                 U g_type_class_adjust_private_offset
                 U g_type_class_peek_parent
                 U g_type_from_name
                 U g_type_interface_peek
                 U g_type_name
                 U g_type_query
                 U g_type_register_static
                 U g_type_register_static_simple
                 U g_udev_client_new
                 U g_udev_client_query_by_subsystem
                 U g_udev_device_get_device_file
                 U g_udev_device_get_property
                 U g_udev_device_get_property_as_int
                 U g_udev_device_get_sysfs_path
                 U g_value_copy
                 U g_value_dup_string
                 U g_value_get_boolean
                 U g_value_get_enum
                 U g_value_get_int
                 U g_value_get_string
                 U g_value_get_uint
                 U g_value_init
                 U g_value_set_boolean
                 U g_value_set_enum
                 U g_value_set_flags
                 U g_value_set_int
                 U g_value_set_string
                 U g_value_set_uint
                 U g_value_take_string
                 U g_value_transform
                 U g_value_unset
                 U gst_allocator_get_type
                 U gst_atomic_queue_length
                 U gst_atomic_queue_new
                 U gst_atomic_queue_pop
                 U gst_atomic_queue_push
                 U gst_atomic_queue_unref
                 U gst_base_sink_wait_preroll
                 U gst_base_src_get_allocator
                 U gst_base_src_get_buffer_pool
                 U gst_base_src_negotiate
                 U gst_base_src_set_caps
                 U gst_base_src_set_format
                 U gst_base_src_set_live
                 U gst_base_transform_get_buffer_pool
                 U gst_base_transform_get_type
                 U gst_base_transform_is_passthrough
                 U gst_base_transform_set_passthrough
                 U gst_base_transform_set_qos_enabled
                 U gst_buffer_add_video_meta_full
                 U gst_buffer_append_memory
                 U gst_buffer_copy_into
                 U gst_buffer_copy_region
                 U gst_buffer_fill
                 U gst_buffer_get_size
                 U gst_buffer_get_video_meta
                 U gst_buffer_map
                 U gst_buffer_n_memory
                 U gst_buffer_new
                 U gst_buffer_new_allocate
                 U gst_buffer_peek_memory
                 U gst_buffer_pool_acquire_buffer
                 U gst_buffer_pool_config_add_option
                 U gst_buffer_pool_config_get_allocator
                 U gst_buffer_pool_config_get_params
                 U gst_buffer_pool_config_has_option
                 U gst_buffer_pool_config_set_allocator
                 U gst_buffer_pool_config_set_params
                 U gst_buffer_pool_config_validate_params
                 U gst_buffer_pool_get_config
                 U gst_buffer_pool_get_type
                 U gst_buffer_pool_is_active
                 U gst_buffer_pool_set_active
                 U gst_buffer_pool_set_config
                 U gst_buffer_pool_set_flushing
                 U gst_buffer_resize
                 U gst_buffer_unmap
                 U gst_caps_append
                 U gst_caps_append_structure
                 U gst_caps_append_structure_full
                 U gst_caps_can_intersect
                 U gst_caps_copy_nth
                 U gst_caps_features_copy
                 U gst_caps_features_is_any
                 U gst_caps_features_is_equal
                 U gst_caps_features_new
                 U gst_caps_fixate
                 U gst_caps_foreach
                 U gst_caps_get_features
                 U gst_caps_get_size
                 U gst_caps_get_structure
                 U gst_caps_intersect
                 U gst_caps_intersect_full
                 U gst_caps_is_any
                 U gst_caps_is_empty
                 U gst_caps_is_equal
                 U gst_caps_is_fixed
                 U gst_caps_is_subset
                 U gst_caps_is_subset_structure_full
                 U gst_caps_map_in_place
                 U gst_caps_merge
                 U gst_caps_new_empty
                 U gst_caps_normalize
                 U gst_caps_set_features
                 U gst_caps_set_value
                 U gst_caps_simplify
                 U gst_caps_truncate
                 U gst_clock_get_time
                 U gst_color_balance_channel_get_type
                 U gst_color_balance_get_type
                 U gst_core_error_quark
                 U gst_debug_log
                 U gst_device_get_type
                 U gst_device_provider_class_set_static_metadata
                 U gst_device_provider_device_add
                 U gst_device_provider_device_remove
                 U gst_device_provider_get_type
                 U gst_device_provider_register
                 U gst_dmabuf_allocator_new
                 U gst_dmabuf_memory_get_fd
                 U gst_element_class_add_pad_template
                 U gst_element_class_set_metadata
                 U gst_element_class_set_static_metadata
                 U gst_element_factory_make
                 U gst_element_get_type
                 U gst_element_message_full
                 U gst_element_post_message
                 U gst_element_register
                 U gst_fd_allocator_alloc
                 U gst_flow_get_name
                 U gst_is_dmabuf_memory
                 U gst_memory_get_sizes
                 U gst_memory_init
                 U gst_memory_is_type
                 U gst_memory_resize
                 U gst_message_new_qos
                 U gst_mini_object_copy
                 U gst_mini_object_get_qdata
                 U gst_mini_object_is_writable
                 U gst_mini_object_make_writable
                 U gst_mini_object_ref
                 U gst_mini_object_replace
                 U gst_mini_object_set_qdata
                 U gst_mini_object_steal_qdata
                 U gst_mini_object_unref
                 U gst_object_get_name
                 U gst_object_ref
                 U gst_object_ref_sink
                 U gst_object_sync_values
                 U gst_object_unref
                 U gst_pad_get_allowed_caps
                 U gst_pad_get_current_caps
                 U gst_pad_get_pad_template_caps
                 U gst_pad_get_task_state
                 U gst_pad_pause_task
                 U gst_pad_peer_query
                 U gst_pad_peer_query_caps
                 U gst_pad_query_caps
                 U gst_pad_start_task
                 U gst_pad_stop_task
                 U gst_pad_template_new
                 U gst_plugin_add_dependency
                 U gst_plugin_register_static
000000000000aa50 T gst_plugin_video4linux2_get_desc
000000000000aa60 T gst_plugin_video4linux2_register
                 U gst_poll_add_fd
                 U gst_poll_fd_ctl_pri
                 U gst_poll_fd_ctl_read
                 U gst_poll_fd_ctl_write
                 U gst_poll_fd_has_error
                 U gst_poll_fd_has_pri
                 U gst_poll_fd_init
                 U gst_poll_free
                 U gst_poll_new
                 U gst_poll_set_flushing
                 U gst_poll_wait
                 U gst_push_src_get_type
                 U gst_query_add_allocation_meta
                 U gst_query_add_allocation_param
                 U gst_query_add_allocation_pool
                 U gst_query_find_allocation_meta
                 U gst_query_get_n_allocation_params
                 U gst_query_get_n_allocation_pools
                 U gst_query_new_allocation
                 U gst_query_parse_allocation
                 U gst_query_parse_caps
                 U gst_query_parse_nth_allocation_meta
                 U gst_query_parse_nth_allocation_param
                 U gst_query_parse_nth_allocation_pool
                 U gst_query_set_caps_result
                 U gst_query_set_latency
                 U gst_query_set_nth_allocation_param
                 U gst_query_set_nth_allocation_pool
                 U gst_resource_error_quark
                 U gst_static_caps_get
                 U gst_stream_error_quark
                 U gst_structure_copy
                 U gst_structure_fixate_field
                 U gst_structure_fixate_field_nearest_fraction
                 U gst_structure_fixate_field_nearest_int
                 U gst_structure_fixate_field_string
                 U gst_structure_foreach
                 U gst_structure_free
                 U gst_structure_get_fraction
                 U gst_structure_get_int
                 U gst_structure_get_name
                 U gst_structure_get_string
                 U gst_structure_get_uint
                 U gst_structure_get_value
                 U gst_structure_has_field
                 U gst_structure_has_name
                 U gst_structure_map_in_place
                 U gst_structure_n_fields
                 U gst_structure_new
                 U gst_structure_new_empty
                 U gst_structure_nth_field_name
                 U gst_structure_remove_field
                 U gst_structure_remove_fields
                 U gst_structure_set
                 U gst_structure_set_value
                 U gst_structure_take_value
                 U gst_structure_to_string
                 U gst_type_mark_as_plugin_api
                 U gst_uri_error_quark
                 U gst_uri_handler_get_type
                 U gst_util_fraction_multiply
                 U gst_util_greatest_common_divisor
                 U gst_util_uint64_scale_int
                 U gst_value_compare
                 U gst_value_fraction_subtract
                 U gst_value_get_fraction_denominator
                 U gst_value_get_fraction_numerator
                 U gst_value_get_structure
                 U gst_value_intersect
                 U gst_value_is_fixed
                 U gst_value_list_append_and_take_value
                 U gst_value_list_append_value
                 U gst_value_list_get_size
                 U gst_value_list_get_value
                 U gst_value_set_fraction
                 U gst_value_set_fraction_range_full
                 U gst_value_set_int_range_step
                 U gst_value_set_structure
                 U gst_value_subtract
                 U gst_video_alignment_reset
                 U gst_video_calculate_display_ratio
                 U gst_video_codec_frame_unref
                 U gst_video_codec_state_ref
                 U gst_video_codec_state_unref
                 U gst_video_colorimetry_is_equal
                 U gst_video_colorimetry_to_string
                 U gst_video_decoder_drop_frame
                 U gst_video_decoder_finish_frame
                 U gst_video_decoder_get_buffer_pool
                 U gst_video_decoder_get_frame
                 U gst_video_decoder_get_oldest_frame
                 U gst_video_decoder_get_type
                 U gst_video_decoder_negotiate
                 U gst_video_decoder_proxy_getcaps
                 U gst_video_decoder_set_latency
                 U gst_video_decoder_set_output_state
                 U gst_video_decoder_set_packetized
                 U gst_video_encoder_allocate_output_buffer
                 U gst_video_encoder_finish_frame
                 U gst_video_encoder_get_frame
                 U gst_video_encoder_get_oldest_frame
                 U gst_video_encoder_get_output_state
                 U gst_video_encoder_get_type
                 U gst_video_encoder_negotiate
                 U gst_video_encoder_set_latency
                 U gst_video_encoder_set_output_state
                 U gst_video_format_to_string
                 U gst_video_frame_copy
                 U gst_video_frame_map
                 U gst_video_frame_unmap
                 U gst_video_info_align_full
                 U gst_video_info_from_caps
                 U gst_video_info_init
                 U gst_video_info_set_interlaced_format
                 U gst_video_info_to_caps
                 U gst_video_interlace_mode_to_string
                 U gst_video_meta_api_get_type
                 U gst_video_meta_get_plane_height
                 U gst_video_meta_set_alignment
                 U gst_video_orientation_get_type
                 U gst_video_sink_get_type
                 U ioctl@GLIBC_2.17
                 U memcpy@GLIBC_2.17
                 U memset@GLIBC_2.17
                 U mmap64@GLIBC_2.17
                 U munmap@GLIBC_2.17
                 U open64@GLIBC_2.17
                 U read@GLIBC_2.17
                 U stat64@GLIBC_2.33
                 U strcmp@GLIBC_2.17
                 U strcpy@GLIBC_2.17
                 U strerror@GLIBC_2.17
                 U strlen@GLIBC_2.17
                 U strncpy@GLIBC_2.17

As the last step it is frustruating to see Gstreamer1.0 cannot find v4l2h264enc or any other encoding elements. The output of

root@linaro-developer:/home/linaro/gstreamer/build/subprojects/gst-plugins-good/sys/v4l2# gst-inspect-1.0 v4l2h264enc
No such element or plugin 'v4l2h264enc'

So where I am missing, any help, comment, or guidance will be appreciated a lot.


Solution

  • So, after grinding a lot I realized OpenMAX might not be the exact module in kernel for hw accelerated video encoding/decoding in Qualcomm boards, to at least in RB5. I inspected different kernel versions and realized only reference to h264 is in the Venus driver. In this page it seems it also supports sm8250. I still couldnt figure out how to connect Venus drivers to GStreamer1.0, I need to dig GStreamer source code more.