opencvffmpegalsa

OpenCV Python and FFmpeg video and audio sync problem


I was trying to use opencv to read frames from capture device , process them and then send them to the standard output (stdout) using the following Python code:

import cv2
import numpy as np
import subprocess
import sys


def process_frame(frame):
    // some frame processing work here 


    sys.stdout.write(frame.tobytes())
    sys.stdout.flush()


#input stream
input_dev = "/dev/video0"


# Open the input video stream
input_stream = cv2.VideoCapture(input_dev)

if not input_stream.isOpened():
    print("Error: Could not open input stream.")
    exit()

input_stream.set(cv2.CAP_PROP_FRAME_WIDTH, 1280)
input_stream.set(cv2.CAP_PROP_FRAME_HEIGHT, 720)

while True:
    ret, frame = input_stream.read()
    if not ret:
        break

    # Process the frame and send it to standard output (stdout)
    process_frame(frame)

# Release the video stream
input_stream.release()

after that i read the output using ffmpeg , and add audio from alsa device :

python ocv.py | ffmpeg \
-f alsa -thread_queue_size 1024 -ac 2 -ss 13 -i hw:2,0 \
-f rawvideo -pixel_format bgr24 -video_size 1280x720 -framerate 20 -thread_queue_size 1024 -i - \
-pix_fmt yuv420p -filter:v scale=in_color_matrix=auto:in_range=auto:out_color_matrix=bt709:out_range=tv \
-colorspace:v bt709 -color_primaries:v bt709 -color_trc:v bt709 -color_range:v tv \
-c:v libx264 -aspect 16/9 -b:v 1M -maxrate 2M -bufsize 1M -preset veryfast \
-tune fastdecode -tune zerolatency -c:a aac -b:a 128k  -f mpegts udp://224.0.0.1:1206?pkt_size=1316

as i excepted there was audio/video sync problem as the audio is about 8 sec ahead video ,maybe due to frame processing using OpenCV .

tried :

-filter:a "adelay=8000|8000"
-itsoffset 8.0 (before the audio input) 

but non of them seems to work or maybe I'm using the wrong form of the command.

can anyone please guide to a method where audio and video are auto synchronized if there is such thing? and if not what is the proper form of the command to make audio delay with about 8 sec ?


Solution

  • Have you tried using the -use_wallclock_as_timestamps option ? the pts will then be assigned to the clock of the system