python-3.xopencvffmpegstdoutv4l2loopback

How to use stdout.write in python3 when piping to ffmpeg?


My main goal for this project I'm working on is to use a python script to get any webcam footage, edit it with opencv, and then pipe the edited video frames with ffmpeg to a dummy webcam from v4l2loopback. Here's the sample code I made that runs exactly as I want to on python 2.7:

import cv2
import subprocess as sp
import sys

cap = cv2.VideoCapture(1)

cv2.namedWindow('result', cv2.WINDOW_AUTOSIZE)

while True:
    ret, frame = cap.read()
    cv2.imshow('result', frame)

    sys.stdout.write(frame.tostring())

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()

and then run it with

python pySample.py | ffmpeg -f rawvideo -pixel_format bgr24 -video_size 640x480 -framerate 30 -i - -vf format=yuv420p -f v4l2 /dev/video17

However, I want it to work with python3 instead of 2.7, and I found a way to make it work where I replace the "sys.stdout..." line with

sys.stdout.buffer.write(frame.tobytes())

This works well, except that it only runs at 14 fps while the 2.7 code could run at 30. I'm kind of at a loss as to how to fix this issue/ what this issue exactly is. I'm running this on a raspberry pi if that helps. Thanks so much!


Solution

  • How to use stdout.write in python3 when piping to ffmpeg?

    as your question is titled "How to use stdout.write in python3 when piping to ffmpeg?", I'm going to answer that first:

    sys.stdout.buffer.write(data)
    

    this is how you do it.

    you already know that (as i've taken the answer from your question), so I guess this is not what you are really asking.

    so your real question seems to be:

    How to write to stdout fast?

    however, this implies that you think that the writing to stdout is slow. why? (most likely because the only line you changed dealt with writing to stdout).

    let's check (using a profiler), where your python-script spends time doing things:

    python3 -m cProfile -o pySample.prof pySample.py | ffmpeg -f rawvideo -pixel_format bgr24 -video_size 640x480 -framerate 30 -i - -vf format=yuv420p -f v4l2 /dev/video17
    

    this creates a pySample.prof file containing all the call information. we can inspect it:

    import pstats
    pstats.Stats("pySample.prof").sort_stats(pstats.SortKey.TIME).print_sorted(5)
    

    this will give us the 5 functions that consumed most time when running the script. for me this returns:

    Mon Nov 16 14:40:40 2020    pySample.prof
    
             70698 function calls (68335 primitive calls) in 49.782 seconds
    
       Ordered by: internal time
       List reduced from 881 to 5 due to restriction <5>
    
       ncalls  tottime  percall  cumtime  percall filename:lineno(function)
          490   40.614    0.083   40.614    0.083 {method 'read' of 'cv2.VideoCapture' objects}
          490    3.813    0.008    3.813    0.008 {method 'write' of '_io.BufferedWriter' objects}
          490    2.334    0.005    2.334    0.005 {waitKey}
          490    1.238    0.003    1.238    0.003 {method 'tobytes' of 'numpy.ndarray' objects}
            1    0.913    0.913   49.783   49.783 pySample.py:1(<module>)
    

    now this is interesting. it basically tells us, that python spent a lot of time reading the data from the video device, and only very little time writing it to the output (and converting it to bytes).

    so your question should really be: how do i speed up video grabbing with OpenCV.

    unfortunately, I can't answer that one ;-)