fixing (with ffmpeg) the chrominance position on a video after capturing

I'm trying to convert some video from VHS to digital using an (old) video capture card (and obviously an old VHS player). Due to the input from my video capture card and the output available from the VHS, I have no other choice than capture with an S-Video cable to a computer.

Almost everything works except a little mis-synchronization between chroma and luma which do not happen on TV.

For example, in the original video, I have something like that:

After capturing the video looks like this:

As you may see, there is a little desynchronization of the chroma with the luma channel (I will say about 10 lines errors).

I'm capturing with ffmpeg on a Linux system with the following commands:

$ v4lctl setnorm PAL-BG

$ v4lctl setinput S-video

$ ffmpeg -y -f alsa -ac 2 -i pulse -f video4linux2 -i /dev/video0 -c:a pcm_s16le -vcodec rawvideo -t $duration -r 25 -loglevel error -stats ~/tmp/tmp.mkv

I tried other input norm in v4l, tried an other VHS player, tried an other conversion cable from SCART to S-Video but it didn't change anything,

My question is simple: is there a way to fix this with a post-processing video filter in ffmpeg?

I already looked at the long list of video filter available in ffmpeg but I didn't find anything.

Also, please note I can't apply filter during the capture commands (old capture cards, old cpu, ..), this is why I capture in rawvideo and native audio. When the capture is done I convert the video/audio into h264/vorbis, at this step I can apply as much as audio/video filtering needed (even if it include extracting chroma & luma to new files, fixing and merging again).

Thanks!

The basic workflow to accomplish this is first, to extract the luma and chroma planes from the source video. Then extract a single corresponding frame from each of the isolated planes, import them in an image editor and find out the displacement of the chroma planes relative to the luma plane. Then lop off the head of the chroma planes based on that information. Merge the modified planes to form the corrected composite video.

Besides FFmpeg, you'll need a tool like dd (for trimming bytes from chroma planes).

Step 1 Extract planes from rawvideo stream

ffmpeg -i in.mkv -filter_complex "[0:v]extractplanes=y+u+v[y][u][v]" \
       -map "[y]" in.y -map "[u]" in.u -map "[v]" in.v

Step 2 Extract frame for examination

ffmpeg -f rawvideo -video_size 720x576 -framerate 25 -pix_fmt gray -i in.y -vframes 1 y.png


ffmpeg -f rawvideo -video_size 360x288 -framerate 25 -pix_fmt gray -i in.u -vframes 1 u.png


ffmpeg -f rawvideo -video_size 360x288 -framerate 25 -pix_fmt gray -i in.v -vframes 1 v.png

The captured video is YUV 4:2:0, so the chroma planes have dimensions half the size.

Step 3 Examine frames in a layer-based image editor

Stack the luma image at the bottom and the two chroma images on top. The chroma images have to be scaled to double the size. Using blend modes or opacity, figure the offsets needed to align the chroma planes with the luma. The values may not be identical for the two chroma images.

Step 4 Use a tool like dd to trim the chroma streams.

Since chroma planes are half-height, the offsets found in the previous step will have to be halved and rounded to an integer.

Let's say the offsets were 6 for U and 8 for V So,

dd if=in.u of=in+6.u bs=360 skip=3

dd if=in.v of=in+8.v bs=360 skip=4

This step assumes the chroma is always delayed relative to the luma, which should be a safe assumption for VCR captures. For academic interest, the reverse case would need the use of dd to extract the required number of lines from a stream, and then a tool like cat to attach it to the head of that stream.

Step 5 Merge planes

ffmpeg -f rawvideo -framerate 25 -pix_fmt gray -video_size 720x576 -i in.y \
       -f rawvideo -framerate 25 -pix_fmt gray -video_size 360x288 -i in+6.u \
       -f rawvideo -framerate 25 -pix_fmt gray -video_size 360x288 -i in+8.v \
       -i in.mkv
       -filter_complex "[0][1][2]mergeplanes=0x001020:yuv420p[v]" \
       -map "[v]" -map 3:a -c:v rawvideo -c:a copy corrected.mkv

Because some lines were chopped off from the chroma planes, the very last frame will not be encoded, since the chroma components will be truncated.