Populated list contains only copies of same element rather than different ones

I've got a problem with method i wrote. It's supposed to extract video frames from mp4 file and write them to list of BufferedImages. Inside while loop every image is different (i checked it by saving every frame extracted from list to file), but outside of the loop, all images in the list are identical.

That's how the method looks like:

private List<BufferedImage> getVideoFrames() {
        List<BufferedImage> frames = new ArrayList<>();
        try (Java2DFrameConverter frameConverter = new Java2DFrameConverter()) {
            int pendingFrame = 0;
            int parsedFrame = 0;
            int skipFrames = (int) frameGrabber.getFrameRate() / 15;

            while (parsedFrame < OUTPUT_IMAGE_FRAME_QUANTITY) {
                Frame frame = frameGrabber.grabFrame(false, true, true, false, false);
                if (pendingFrame % skipFrames == 0) {
                    frames.add(frameConverter.convert(frame));
                    parsedFrame++;
                }
                pendingFrame++;
            }

            frameGrabber.stop();
        } catch (FFmpegFrameGrabber.Exception e) {
            logger.error(e.getMessage());
        }

        return frames;
    }

I don't know what I'm doing wrong. This behaviour is somewhat surprising to me.

Solution

Refer to the source of FFmpegFrameGrabber.java to figure this one out. Specifically, the grabFrame method (the link goes to the right method automatically).

Note how there is no new anywhere in that method. It just starts updating the fields in frame (e.g. line 1391, frame.keyFrame = ...).

That's because frame is a field, defined in line 391.

That field is initialized only once, ever in the start procedure (line 925).

Conclusion:

There is only ever a single frame object.
The grabFrame method merely updates this single object. It always returns the same reference - a pointer to that one frame object that any given grabber only ever makes once.
There is no functional difference between calling grabFrame once, saving what it returns (which is the one and only Frame object), and then calling grabFrame a few more times, ignoring what it returns. Because it never makes new frames, it merely updates the existing frame object and returns a reference to it over and over again.

What's happening is similar to this:

void example() {
  AtomicInteger i = new AtomicInteger();
  var ints = new ArrayList<AtomicInteger>();
  for (int i = 0; i < 10; i++) {
    i.incrementAndGet();
    ints.add(i);
  }

  ints.forEach(System.out::println);
}

Which prints 10, ten times (because there's just a single AtomicInteger object, and a list with a pointer to the same object 10 times. It's like an address book with 10 pages, each page with the identical street address on it).

Fixing it

When saving the frame data to your list, make a copy.

Alternatively, because you're doing video, keeping track of every frame is incredibly expensive. Generally frames are uncompressed, meaning, for your average 4k video, that's 4096 x 2160 x 4 = 8847364 bytes of data, and you get 30 to 60 of those a second, for 265420920 bytes per second - go through 5 minutes and you end up with 79626276000 bytes - 75GB, far more than your RAM can generally hold.

Therefore instead, the general 'flow' of video-related apps is to capture a frame, process it, and then get rid of it - do not keep a list of frames to operate on, instead process each frame as it comes in so you don't have to keep it around.

That's clearly how the FFmpegFrameGrabber code is designed to be used, which is why it just overwrites the same frame object every time you call grabFrame.