The short version: Using piped output from ffmpeg produces a file with incorrect metadata.
ffmpeg -y -i .\test_mp4.mp4 -f avi -c:v libx264 - > output.avi to make an AVI file using the pipe output.
ffprobe -v error -count_frames -show_entries stream=duration,nb_read_frames,r_frame_rate .\output.avi
The output will show that the metadata does not match the actual frames contained in the video.
Details below.
Using Python, I am attempting to use ffmpeg to compress videos and put them in a PowerPoint. This works great, however, the video files themselves have incorrect frame counts which can cause issues when I read from those videos in other code.
Edit for clarification: by "frame count" I mean the metadata frame count. The actual number of frames contained in the video is correct, but querying the metadata gives an incorrect frame count.
Having eliminated the PowerPoint aspect of the code, I've narrowed this down to the following minimal reproducing example of saving an output from an ffmpeg pipe:
from subprocess import Popen, PIPE
video_path = 'test_mp4.mp4'
ffmpeg_pipe = Popen(['ffmpeg',
'-y', # Overwrite files
'-i', f'{video_path}', # Input from file
'-f', 'avi', # Output format
'-c:v', 'libx264', # Codec
'-'], # Output to pipe
stdout=PIPE)
new_path = "piped_video.avi"
vid_file = open(new_path, "wb")
vid_file.write(ffmpeg_pipe.stdout.read())
vid_file.close()
I've tested several different videos. One small example video that I've tested can be found here.
I've tried a few different codecs with avi format and tried libvpx with webm format. For the avi outputs, the frame count usually reads as 1073741824 (2^30). Weirdly, for the webm format, the frame count read as -276701161105643264.
Edit: This issue can also be reproduced with just ffmpeg in command prompt using the following command:
ffmpeg -y -i .\test_mp4.mp4 -f avi -c:v libx264 - > output.avi
This is a snippet I used to read the frame count, but one could also see the error by opening the video details in Windows Explorer and seeing the total time as something like 9942 hours, 3 minutes, and 14 seconds.
import cv2
video_path = 'test_mp4.mp4'
new_path = "piped_video.webm"
cap = cv2.VideoCapture(video_path)
print(f"Original video frame count: = {int(cap.get(cv2.CAP_PROP_FRAME_COUNT)):d}")
cap.release()
cap = cv2.VideoCapture(new_path)
print(f"Piped video frame count: = {int(cap.get(cv2.CAP_PROP_FRAME_COUNT)):d}")
cap.release()
The error can also be observed using ffprobe with the following command: ffprobe -v error -count_frames -show_entries stream=duration,nb_read_frames,r_frame_rate .\output.avi. Note that the frame rate and number of frames counted by ffprobe do not match with the duration from the metadata.
For completeness, here is the ffmpeg output:
ffmpeg version 2023-06-11-git-09621fd7d9-full_build-www.gyan.dev Copyright (c) 2000-2023 the FFmpeg developers
built with gcc 12.2.0 (Rev10, Built by MSYS2 project)
configuration: --enable-gpl --enable-version3 --enable-static --disable-w32threads --disable-autodetect --enable-fontconfig --enable-iconv --enable-gnutls --enable-libxml2 --enable-gmp --enable-bzlib --enable-lzma --enable-libsnappy --enable-zlib --enable-librist --enable-libsrt --enable-libssh --enable-libzmq --enable-avisynth --enable-libbluray --enable-libcaca --enable-sdl2 --enable-libaribb24 --enable-libaribcaption --enable-libdav1d --enable-libdavs2 --enable-libuavs3d --enable-libzvbi --enable-librav1e --enable-libsvtav1 --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxavs2 --enable-libxvid --enable-libaom --enable-libjxl --enable-libopenjpeg --enable-libvpx --enable-mediafoundation --enable-libass --enable-frei0r --enable-libfreetype --enable-libfribidi --enable-liblensfun --enable-libvidstab --enable-libvmaf --enable-libzimg --enable-amf --enable-cuda-llvm --enable-cuvid --enable-ffnvcodec --enable-nvdec --enable-nvenc --enable-d3d11va --enable-dxva2 --enable-libvpl --enable-libshaderc --enable-vulkan --enable-libplacebo --enable-opencl --enable-libcdio --enable-libgme --enable-libmodplug --enable-libopenmpt --enable-libopencore-amrwb --enable-libmp3lame --enable-libshine --enable-libtheora --enable-libtwolame --enable-libvo-amrwbenc --enable-libcodec2 --enable-libilbc --enable-libgsm --enable-libopencore-amrnb --enable-libopus --enable-libspeex --enable-libvorbis --enable-ladspa --enable-libbs2b --enable-libflite --enable-libmysofa --enable-librubberband --enable-libsoxr --enable-chromaprint
libavutil 58. 13.100 / 58. 13.100
libavcodec 60. 17.100 / 60. 17.100
libavformat 60. 6.100 / 60. 6.100
libavdevice 60. 2.100 / 60. 2.100
libavfilter 9. 8.101 / 9. 8.101
libswscale 7. 3.100 / 7. 3.100
libswresample 4. 11.100 / 4. 11.100
libpostproc 57. 2.100 / 57. 2.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'test_mp4.mp4':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: isommp42
creation_time : 2022-08-10T12:54:09.000000Z
Duration: 00:00:06.67, start: 0.000000, bitrate: 567 kb/s
Stream #0:0[0x1](eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p(progressive), 384x264 [SAR 1:1 DAR 16:11], 563 kb/s, 30 fps, 30 tbr, 30k tbn (default)
Metadata:
creation_time : 2022-08-10T12:54:09.000000Z
handler_name : Mainconcept MP4 Video Media Handler
vendor_id : [0][0][0][0]
encoder : AVC Coding
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> h264 (libx264))
Press [q] to stop, [?] for help
[libx264 @ 0000018c68c8b9c0] using SAR=1/1
[libx264 @ 0000018c68c8b9c0] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2
[libx264 @ 0000018c68c8b9c0] profile High, level 2.1, 4:2:0, 8-bit
Output #0, avi, to 'pipe:':
Metadata:
major_brand : mp42
minor_version : 0
compatible_brands: isommp42
ISFT : Lavf60.6.100
Stream #0:0(eng): Video: h264 (H264 / 0x34363248), yuv420p(progressive), 384x264 [SAR 1:1 DAR 16:11], q=2-31, 30 fps, 30 tbn (default)
Metadata:
creation_time : 2022-08-10T12:54:09.000000Z
handler_name : Mainconcept MP4 Video Media Handler
vendor_id : [0][0][0][0]
encoder : Lavc60.17.100 libx264
Side data:
cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: N/A
[out#0/avi @ 0000018c687f47c0] video:82kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 3.631060%
frame= 200 fps=0.0 q=-1.0 Lsize= 85kB time=00:00:06.56 bitrate= 106.5kbits/s speed=76.2x
[libx264 @ 0000018c68c8b9c0] frame I:1 Avg QP:16.12 size: 3659
[libx264 @ 0000018c68c8b9c0] frame P:80 Avg QP:21.31 size: 647
[libx264 @ 0000018c68c8b9c0] frame B:119 Avg QP:26.74 size: 243
[libx264 @ 0000018c68c8b9c0] consecutive B-frames: 3.0% 53.0% 0.0% 44.0%
[libx264 @ 0000018c68c8b9c0] mb I I16..4: 17.6% 70.6% 11.8%
[libx264 @ 0000018c68c8b9c0] mb P I16..4: 0.8% 1.7% 0.6% P16..4: 17.6% 4.6% 3.3% 0.0% 0.0% skip:71.4%
[libx264 @ 0000018c68c8b9c0] mb B I16..4: 0.1% 0.3% 0.2% B16..8: 11.7% 1.4% 0.4% direct: 0.6% skip:85.4% L0:32.0% L1:59.7% BI: 8.3%
[libx264 @ 0000018c68c8b9c0] 8x8 transform intra:59.6% inter:62.4%
[libx264 @ 0000018c68c8b9c0] coded y,uvDC,uvAC intra: 48.5% 0.0% 0.0% inter: 3.5% 0.0% 0.0%
[libx264 @ 0000018c68c8b9c0] i16 v,h,dc,p: 19% 39% 25% 17%
[libx264 @ 0000018c68c8b9c0] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 21% 25% 30% 3% 3% 4% 4% 4% 5%
[libx264 @ 0000018c68c8b9c0] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 22% 20% 16% 6% 8% 8% 8% 5% 6%
[libx264 @ 0000018c68c8b9c0] i8c dc,h,v,p: 100% 0% 0% 0%
[libx264 @ 0000018c68c8b9c0] Weighted P-Frames: Y:0.0% UV:0.0%
[libx264 @ 0000018c68c8b9c0] ref P L0: 76.2% 7.9% 11.2% 4.7%
[libx264 @ 0000018c68c8b9c0] ref B L0: 85.6% 12.9% 1.5%
[libx264 @ 0000018c68c8b9c0] ref B L1: 97.7% 2.3%
[libx264 @ 0000018c68c8b9c0] kb/s:101.19
So the question is: why does this happen, and how can one avoid it?
As I commented above, it is 100% due to outputting AVI file to a pipe. Check out this part of FFmpeg source code:
Specifically, the if block starting on Line 924 is skipped if you write to a pipe:
if (pb->seekable & AVIO_SEEKABLE_NORMAL) {
...
if (avi->riff_id == 1) {
...
} else {
...
avio_wl32(pb, nb_frames);
...
}
}
This cause the piped output to miss some header attributes, including nb_frames as the excerpts above indicates. In addition, some stream attributes are skipped as well (Lines 969-)
So, what you are experiencing is indeed intentional, and it's highly unlikely FFmpeg devs will consider this as a bug.
I was to offer a Python script to fill the nb_frames manually to the retrieved bytearray, but other skipped fields may cause issues anyway. So, I suggest you to just write the AVI file (MP4 file is probably a better choice for PowerPoint nowadays, BTW) to a temp dir and read the output file. Something like this:
from tempfile import TemporaryDirectory
import subprocess as sp
from os import path
video_path = 'test_mp4.mp4'
with TemporaryDirectory() as temp_dir:
new_path = path.join(temp_dir, "piped_video.avi")
sp.run('ffmpeg',
'-y', # Overwrite files
'-i', f'{video_path}', # Input from file
'-f', 'avi', # Output format
'-c:v', 'libx264', # Codec
new_path]) # Output to pipe
from open(new_path,'rb') as f:
b = f.read()