imagevideoffmpegjavacv

Create a video form images and videos and add audio to it programatically | Java, Python, ffmpeg


I want to create a new video (finalVideo.mp4) with:

  1. 3 image files image1.jpg, image2.jpg, image3.jpg, each image to be seen for 4 seconds
  2. 1 video file (video1.mp4), lasts for about 30 seconds with a frame rate of 30fps
  3. 1 audio file (audio1.mp3)

In the final video (finalVideo.mp4) I want the audio (audio1.mp3) to play only when the images are present and the video's (video1.mp4) audio when the video is present.

For example: The final video (finalVideo.mp4) contains image1.jpg, video1.mp4, image2.jpg and image3.jpg (in order). The audio (audio1.mp3) should play for the first 4 seconds, then the video's (video1.mp4) audio for the next 30 seconds and then the audio (audio1.mp3) for the next 8 (4*2) seconds.

Please let me know how to do this programmatically. I was hoping to figure this out using Java (JavaCV) or Python (OpenCV). But there is no programming language restriction as such, the answer can be in any language.


EDIT 1

Here is my attempt at making this work:

String ffmpeg = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
        ProcessBuilder processBuilder = new ProcessBuilder(
                ffmpeg,
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image1.jpg",
                "-i", "video1.mp4",
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image2.jpg",
                "-loop", "1",
                "-framerate", "30",
                "-t", "4",
                "-i", "image3.jpg",
                "-filter_complex", "[0][1][2][3]concat=n=4:v=1:a=0",
                "finalVideo.mp4"
        );
        processBuilder.inheritIO().start().waitFor();

The above code only solves the video creation part without audio. Please let me know how to add audio to this video as stated above.


Solution

  • You can do the same with the audio, use -ss and -t input options to control where to clip it.

    String ffmpeg = Loader.load(org.bytedeco.ffmpeg.ffmpeg.class);
            ProcessBuilder processBuilder = new ProcessBuilder(
                    ffmpeg,
                    "-loop", "1",
                    "-framerate", "30",
                    "-t", "4",
                    "-i", "image1.jpg",
                    "-i", "video1.mp4",
                    "-loop", "1",
                    "-framerate", "30",
                    "-t", "4",
                    "-i", "image2.jpg",
                    "-loop", "1",
                    "-framerate", "30",
                    "-t", "4",
                    "-i", "image3.jpg",
                    "-t" 4, "-i", "audio1.mp3",
                    "-ss" 4, "-t", 8, "audio1.mp3",
                    "-filter_complex", 
                    "[0][1:v][2][[3]concat=n=4:v=1:a=0;[4][1:a][5]concat=n=3:v=0:a=1;",
                    "finalVideo.mp4"
            );
            processBuilder.inheritIO().start().waitFor();
    

    (not 100% if you can use the same input file twice, but this should work.)