Right way to use vmaf with ffmpeg

I am trying to calculate the VMAF score of a processed video wrt the original file.

Command I have used:

ffmpeg -y -loglevel info -stats -i original.mp4 -i processed.mp4 -lavfi "[0]null[refdeint];[refdeint]scale=1920:1080:flags=neighbor[ref];[1]setpts=PTS+0.0/TB[b];[b]scale=1920:1080:flags=neighbor[c];[c][ref]libvmaf=log_fmt=json:phone_model=1:model_path={model_path_here}/vmaf_v0.6.1.json:n_subsample=1:log_path=log.json" -f null -

Now as per the official documentation of vmaf with ffmpeg found here, it says source/reference file followed by the encoded/distorted/processed file.

But almost all of the blogs I came across, they are using the other way round order of the args, i.e. processed file followed by the original file.

Few examples:

https://medium.com/@eyevinntechnology/keep-an-eye-on-the-video-quality-b9bcb58dd5a1: search for "Using VMAF within FFMPEG" in it.
https://websites.fraunhofer.de/video-dev/calculating-vmaf-and-psnr-with-ffmpeg/: search for "Metric Calculation with FFmpeg" in it.

EDIT

NOTE: Changing the order does change the VMAF score.

Solution

Files inputted to ffmpeg are numbered in-order.

With ffmpeg -i zero.yuv -i one.yuv:

[0:v] denotes the video stream for -i zero.yuv.
[1:v] denotes the video stream for -i one.yuv.

However, the libvmaf filter expects input streams in the order [distorted][reference]. By default, the inputs will be [0:v][1:v]. However, you can customize it by explicitly specifying the inputs.

Netflix manual (I removed not needed parts):

ffmpeg ... -i src01_hrc00_576x324.yuv \
       ... -i src01_hrc01_576x324.yuv \
    -lavfi "[0:v]setpts=PTS-STARTPTS[reference]; \
            [1:v]setpts=PTS-STARTPTS[distorted]; \
            [distorted][reference]libvmaf=model_path={your_vmaf_dir}/model/vmaf_v0.6.1.json" \
    -f null -

Notice that video streams were relabeled before being fed into libvmaf:

[0:v]...[reference] relabels -i src01_hrc00_576x324.yuv
[1:v]...[distorted] relabels -i src01_hrc01_576x324.yuv

Then, they were fed into [distorted][reference]libvmaf.

Article on medium:

ffmpeg -i distorted-file -i reference-file -lavfi libvmaf -f null –

Distorted file has index=0
Reference file has index=1.

As there is no any renaming/remapping, they will be sent to libvmaf in original order: 0 then 1 (distorder then reference). It is equal to [0][1]libvmaf. So video streams order supplied to libvmaf is exactly the same as in Netflix manual.

FFMpeg manual:

ffmpeg -i main.mpg -i ref.mpg -lavfi libvmaf -f null -

It is better to keep original order if you compare two videos only as otherwise people might be confused.

However, I do have opposite order in my FFmpeg GUI (FFMetrics). The reason is -- it is possible to specify a multiple distorted files to the program so it is easier to understand and use:

ffmetrics.exe [options] ref distorted1 [distorted2] [distorted3] [distorted4] ...

Your command line is also correct:

ffmpeg -i reference -i distorted [0]...[refdeint];[refdeint]...[ref];[1]...[b];[b]...[c];[c][ref]libvmaf...

After all re-mapping streams supplied to libvmaf in correct order: distorted then reference. You must not swap two "-i file" options there as this will change your logic dramatically.

P.S. This was simplified version of what is actually happening :)