videoffmpegencodingalphahevc

How come the quality of HEVC with alpha from my ffmpeg with videotoolbox is not ok

I'm encoding ProRes 4444 with alpha channel video's to transparent video for web in both VP9 (webm) and HEVC (mp4). I'm using a great free tool by Rotato to encode to both formats at once.

However, because I would like to have more control over e.g. the dimensions and the bitrate, I'm looking into ffmpeg. I can encode to both formats with the following params.

# vp9
ffmpeg -i in.mov -vf scale=-1:720 -c:v libvpx-vp9 -auto-alt-ref 0 -pix_fmt yuva420p -acodec libvorbis -cpu-used 3 -deadline good -crf 40 out.webm 

# hevc
ffmpeg -i in.mov -vf scale=-1:720 -c:v hevc_videotoolbox -allow_sw 1 -alpha_quality 0.9 -tag:v hvc1 -q:v 65 out.mp4

The problem: the videotoolbox encoder generates a weird white garbage around the black pencil animations and fade-in's from transparent to black pencil become fade-in's from white pencil to black pencil!?

Here are screenshots of the two compared (top one is my ffmpeg output):

hevc

rotato

Anyone have good settings for ffmpeg I cano use to encode good quality HEVC with alpha? Hardware: MacBook Air M1 and Win11 + GTX 1060 (I should be able to use NVEnc, but don't know how)

Solution

  • For me, the solution was to use the premultiply filter

    -vf premultiply=inplace=1