javaffmpegjavacv

JavaCV FFmpegFrameGrabber & Java2DFrameConverter creating weird looking image


I'm new to JavaCV, so the issue is probably very obvious. I'm trying to do the easy said, difficult done, task of getting the images and audio from a video so I can start making a video editor. After lots of confusion and errors, I am finally getting a result, but it is as odd as the errors. The image appears to be squished in the x direction, with the extra space to the right being transparent (so the image size matches the video's size). Additionally, it has a lot of extra transparent pixels and is multicolored in an odd way.

What the image should look like: https://gofile.io/d/1lQnNd

What the image looks like: https://gofile.io/d/kc09G7

Here is my code:

try {
    FFmpegFrameGrabber frameGrabber = new FFmpegFrameGrabber("C:/Users/mega12345mega/Desktop/Test Files/video.mp4");
    frameGrabber.setFormat("mp4");
    frameGrabber.start();

    int width = frameGrabber.getImageWidth();
    int height = frameGrabber.getImageHeight();
    System.out.println("width: " + width + ", height: " + height);

    Frame frame = frameGrabber.grabImage();
    if (frame == null)
        throw new Exception("Frame is NULL!");
    if (frame.image == null)
        throw new Exception("Frame Image is NULL!");

    BufferedImage bi = new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);
    if (bi == null)
        throw new Exception("bi is NULL!");

    Java2DFrameConverter.copy(frame, bi);
    ImageIO.write(bi, "png", new File("C:/Users/mega12345mega/Desktop/Test Files/img.png"));

    frameGrabber.stop();
    frameGrabber.close();
} catch (Exception e) {
    throw new Exception("Error Getting Image", e);
}

If you are interested, here is the console:

width: 1280, height: 720
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:/Users/mega12345mega/Desktop/Test Files/video.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf58.29.100
  Duration: 00:00:04.95, start: 0.000000, bitrate: 16375 kb/s
    Stream #0:0(und): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1280x720 [SAR 1:1 DAR 16:9], 16401 kb/s, 60 fps, 60 tbr, 90k tbn, 120 tbc (default)
    Metadata:
      handler_name    : VideoHandler
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 44100 Hz, stereo, fltp, 2 kb/s (default)
    Metadata:
      handler_name    : SoundHandler

Additionally, if I just do frameGrabber.grab() instead of frameGrabber.grabImage() (which I believe leaves out the audio), the frame's image property is null (that is what the frame.image == null statement is there for). I am not sure if that belongs in a new question, but help on that is also appreciated.

Thanks in advance!


Solution

  • I figured out how to fix this, but I'm not sure why this works. (Comments about this are appreciated!)

    After looking at this issue - though the problem was different - I found that they used this in their code:

    bi = new Java2DFrameConverter().convert(frame);
    

    What I did was this:

    Java2DFrameConverter.copy(frame, bi);
    

    For some reason, this fixes the issue.
    Obviously, with the solution, you can remove this line as well:

    new BufferedImage(width, height, BufferedImage.TYPE_INT_ARGB);