How to synchronize dual IP cameras on open cv?

I have an object detection program that sometimes utilizes dual cameras to record.

I had no problem synchronizing the cameras when I used USB webcams, but I have recently switched to RJ45 IP PoE security cameras, and now the recording is up to 2 seconds (45 frames) desynchronized.

My code looks like this:

webcam = cv2.VideoCapture(pipeargs.source)
isvideo = False

if "video" in pipeargs.source:
    isvideo = True
camlog.info('camera created')

if pipeargs.source2 is not None:
    webcam2 = cv2.VideoCapture(pipeargs.source2)

while True:
    stream_ok, frame = webcam.read()
    if pipeargs.source2 is not None:
        stream_ok2, frame2 = webcam2.read()

This perfectly synchronized the USB webcams, but now the source has switched from the cameras ID to an rtsp url:

camera1 = rtsp://192.168.2.32:554/user=admin&password=&channel=1&stream=0.sdp?

;camera1 = 0

Cameras are using h.264 (could use h.265), and I believe the problem resides on cameras bursting many frames in 1 packet?, not sure yet, if that is the case the only way to synchronize would be to somehow know how many frames to discard at the start from the camera that goes faster.

EDIT FOR CLARIFITCATION: The goal is to have 2 frames that are miliseconds appart and can be analyced in combination, because currently they are separated by seconds.

Solution

I have found 2 solutions:

First of all there is an object that I know that can only appear once on the images. Tracking it, there could be 4 states:

-Left camera appeareance

-Right camera appeareance

-Both cameras appearence

-No camera appeareance

Tracking that object I concluded that the desynchronization between videos was sometimes 42FPS and on another set of cameras was 50FPS, so I just dropped a few frames and it went perfect.

The problem is that if the object decided to stay exactly between the cameras for some time it could not be tracked and the method would turn innacurate.

The other solution is more practical, when I request the first frame (or when I do the videocapture) I initiate a streaming proccess on the camera. This proccess blocks the proccess for a couple of seconds until the camera can send me back a packet with frames that I will decode one by one. The problem is that the sequential initiation is nowhere simultaneous. The key is to initiate the cameras on 2 proccess to do it nearly at same time. Once initiated, it doesn't matter anymore.

Problem with this method is that if the camera monitor is running on the browser, the initialization is already running and it doesn't work. Gotta make sure there is no streaming going on from other source.