ValueError: No valid stream found in input file. Is -1 of the desired media type?

While using torchcodec with the following code

import torchcodec
file_path = "/path/to/episode_000000.mp4"
x = torchcodec.decoders.VideoDecoder(file_path, seek_mode="approximate")

I got the following error:

[rank0]:   File "/home/hzxie/Development/vla/utils/datasets.py", line 202, in _get_video_bytes
[rank0]:     x = torchcodec.decoders.VideoDecoder(vbytes)
[rank0]:   File "/home/hzxie/Applications/miniconda3/envs/lerobot/lib/python3.10/site-packages/torchcodec/decoders/_video_decoder.py", line 98, in __init__
[rank0]:     core.add_video_stream(
[rank0]:   File "/home/hzxie/Applications/miniconda3/envs/lerobot/lib/python3.10/site-packages/torch/_ops.py", line 723, in __call__
[rank0]:     return self._op(*args, **kwargs)
[rank0]: ValueError: No valid stream found in input file. Is -1 of the desired media type?

How to solve this problem?

Solution 1

Install ffmpeg==6.1.1

conda install -c conda-forge ffmpeg=6.1.1 -y

Solution 2

Use pyav as video backend as a compomise solution. But it is 40% faster than torchcodec.

import av

container = av.open(path_to_video)
for frame in container.decode(video=0):
    frame.to_image().save('frame-%04d.jpg' % frame.index)

In LeRobot, you can pass the backend="pyav" to decode_video_frames as following:

lerobot.common.datasets.video_utils(..., backend="pyav")