Is it possible to interpolate a quarter of the video with optical flow?

I am now trying to interpolate video using optical flow. I was able to interpolate the video by referring to this question and using it as a reference. So my question is: Is it possible to interpolate the video in 1/4 units even finer using the original frame and the optical flow frame? Thank you in advance.

I tried halving the optical flow frame and remapped it to the original image, but it did not work. (It's obvious, of course, but...).

I will try solving this problem with the help of an example. Suppose my previous image is

and my next image is

The next image was created by translating the previous image towards right (I hope it is visible).

Now, let's calculate the optical flow between the two images and plot the flow vectors.

optical_flow = cv2.calcOpticalFlowFarneback(img1, img2, None, pyr_scale = 0.5, levels = 3, winsize = 200, iterations = 3, poly_n = 5, poly_sigma = 1.1, flags = 0)

I have taken the previous image and drawn flow vectors on top of it. This image shows that, previous image was formed by translating the next image towards the left.

If you followed this, it is pretty easy to accomplish the task at hand .i.e interpolate frames between these two frames

Essentially, optical flow vectors tell us for each pixel, where has the pixel come from the previous image. Lets talk about a particular pixel, which has moved from (100, 100) to (200, 100) .i.e moved in the right direction by 100 pixels. The question is where will this pixel be if there was uniform motion between these 2 frames? Example: if there were 4 frames in between these two frames, the pixel will be at locations

initial location     =  (100, 100)  
interpolated frame 1 =  (120, 100)
interpolated frame 2 =  (140, 100)
interpolated frame 3 =  (160, 100)
interpolated frame 4 =  (180, 100)
final location.      =  (200, 100)

Enough talking, let me show you some code.

def load_image(image_path):
    "Load the image and convert it into a grayscale image"
    img = cv2.imread(image_path)
    return cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)


def plot_flow_vectors(img, flow):
    """
    Plots the flow vectors
    """
    img = img.copy()
    h, w = img.shape[:2]
    step_size = 40
    cv2.imshow("original_image", img)
    for y in range(0, h, step_size):
        for x in range(0, w, step_size):
            # Get the flow vector at this point
            dx, dy = flow[y, x]
            # Draw an arrow to represent the flow vector
            cv2.arrowedLine(img, (x, y), (int(x + dx), int(y + dy)), (0, 255, 0), 1, cv2.LINE_AA, tipLength=0.8)
    cv2.imshow("image_with_flow_vectors", img)
    cv2.waitKey(0)
    cv2.destroyAllWindows()

img1 = load_image('first.jpeg') // load previous image
img2 = load_image('second.jpeg') // load next image
optical_flow = cv2.calcOpticalFlowFarneback(img1, img2, None, pyr_scale = 0.5, levels = 3, winsize = 200, iterations = 3, poly_n = 5, poly_sigma = 1.1, flags = 0) // calculate optical flow

plot_flow_vectors(img1, optical_flow) // plot flow vectors


// Generate frames in between 
num_frames = 10
h, w = optical_flow.shape[:2]
for frame_num in range(1, num_frames+1):
    alpha = frame_num / num_frames
    flow =  alpha * optical_flow
    flow[:,:,0] += np.arange(w)
    flow[:,:,1] += np.arange(h)[:,np.newaxis]
    interpolated_frame = cv2.remap(img2, flow, None, cv2.INTER_LINEAR)
    cv2.imshow("intermediate_frame", interpolated_frame)
    cv2.waitKey(0)

PS: optical flow vectors also show an motion in the y direction (when there isn't). I think it is an artifact of the large motion between the two frames. This is generally not the case for frames in a video.

Edit: I created a github repository to interpolate frames in between two video frames to increase the video fps or get a slow motion video out of the input video. Check it out here