How to correlate a GStreamer buffer timestamp with the Linux system clock

I'm working with a GStreamer-1.0 pipeline that (among other things) reads live video from a camera via a v4l2src element and feeds data into an appsink element. We perform some image processing with the images generated by the appsink, producing data about the contents of the image.

Another part of the application reads data from sensors (e.g. accelerometers).

We need to correlate the timestamps for the data from the two sources. Our sensor data is timestamped with the Linux monotonic clock timestamp. We now need to correlate this with the data generated from the video frame that was processed at the same time.

In our video processing thread, we can get the buffer's timestamp with:

...
g_signal_emit_by_name(appsink, "pull-sample", &sample);
buffer = gst_sample_get_buffer(sample);
timestamp = GST_BUFFER_PTS(buffer);
...

But the GST timestamps do not correlate to the system's clock. They appear to be a time offset since a GST-specific epoch (when the pipeline transitioned to PLAYING, I think).

I can, of course, get the system clock at the same time I read a frame and compute the difference, but I am hoping there is a better way to do this. Perhaps one of:

Get the Linux system monotonic clock time corresponding to GST's zero-timestamp. So I can add this offset to each buffer's timestamp.
Use a function (that I must have missed when reading the documentation) to convert buffer timestamps to a system timestamp.
Configure the pipeline or its elements to use the system monotonic clock as the buffer timestamp.

I've tried several different things that have not worked, including:

Get the system clock (gst_system_clock_obtain) and then setting its clock-type property to MONOTONIC
Do the above, and afterward call gst_pipeline_use_clock to set this as the pipeline's clock
Call gst_base_src_set_do_timestamp on the v4l2src object (tried with both true and false)

Any ideas? Or is the best approach to simply read the system clock at the same time I get a buffer from the appsink and compute my own delta?

Solution

I finally found the solution. Internally, GST elements maintain a base_time. Every time a buffer is generated, a source element reads its clock (usually the same clock shared by the rest of the pipeline) and subtracts the base_time from it.

The base_time is set to the clock's current value when the element transitions to the PLAYING state. So each buffer's timestamp ends up being the time elapsed since the element started playing.

So in order to accomplish my goal, two steps are required:

After pipeline creation, configure it to use a system-monotonic clock instead of some other clock that isn't synchronized with the OS's clock.

Assuming pipe is a pointer to the pipeline, this will set its clock to system-monotonic:
```
GstClock *clock = gst_system_clock_obtain();
g_object_set(clock,
             "clock-type", GST_CLOCK_TYPE_MONOTONIC,
             NULL);
gst_pipeline_use_clock(GST_PIPELINE_CAST(pipe), clock);
gst_object_unref(clock)
```
After the pipeline transitions to the PLAYING state, get the source element's base_time value, which will be the system-monotonic value corresponding to buffer-time zero.

Assuming vsrc is a pointer to the v4l2src object at the start of the pipeline, then the following code (executed after gst_element_set_state successfully sets the pipeline to PLAYING state) will get the source's base_time:
```
GstClockTime base_time;

base_time = gst_element_get_base_time(vsrc);
```

With this value, I can now add the base_time to each buffer's timestamp to get the monotonic time corresponding to the buffer's creation:

...
g_signal_emit_by_name(appsink, "pull-sample", &sample);
buffer = gst_sample_get_buffer(sample);
buffer_ts = GST_BUFFER_PTS(buffer);
monotonic_ts = buffer_ts + base_time;

printf("Buffer TS: %" GST_TIME_FORMAT
       " is monotonic TS: %" GST_TIME_FORMAT "\n",
       GST_TIME_ARGS(buffer_ts),
       GST_TIME_ARGS(monotonic_ts));
...