imagehaskellvideoffmpegjuicy-pixels

Haskell - Turning multiple image-files into one video-file using the ffmpeg-light package


Background
I wrote an application for image-processing which uses the ffmpeg-light package to fetch all the frames of a given video-file so that the program afterwards is able to apply grayscaling, as well as edge detection alogrithms to each of the frames.

Now I'm trying to put all of the frames back into a single video-file.

Used Libs
ffmpeg-light-0.12.0
JuicyPixels-3.2.8.3
...

What have I tried?
I have to be honest, I didn't really try anything because I'm kinda clueless where and how to start. I saw that there is a package called Command which allows running processes/commands using the command line. With that I could use ffmpeg (not ffmpeg-light) to create a video out of image-files which I would have to save to the hard drive first but that would be kinda hacky.

Within the documentation of ffmpeg-light on hackage (ffmpeg-light docu) I found the frameWriter function which sounds promising.

frameWriter :: EncodingParams -> FilePath -> IO (Maybe (AVPixelFormat, V2 CInt, Vector CUChar) -> IO ()) 

I guess FilePath would be the location where the video file gets stored but I can't really imagine how to apply the frames as EncodingParams to this function.

Others
I can access:

Question
Is there a way to achieve this using the ffmpeg-light package?

As the ffmpeg-light package lacks of documentation when it comes to conversion from images to video, I really would appreciate your help. (I do not expect a fully working solution.)

Code
The code that reads the frames:

-- Gets and returns all frames that a given video contains
getAllFrames :: String -> IO [(Double, DynamicImage)]
getAllFrames vidPath = do 
  result <- try (imageReaderTime $ File vidPath) :: IO (Either SomeException (IO (Maybe (Image PixelRGB8, Double)), IO()))
  case result of 
    Left ex -> do 
                 printStatus "Invalid video-path or invalid video-format detected." "Video" 
                 return []
    Right (getFrame, _) -> addNextFrame getFrame [] 

-- Adds up all available frames to a video.
addNextFrame :: IO (Maybe (Image PixelRGB8, Double)) -> [(Double, DynamicImage)] -> IO [(Double, DynamicImage)]
addNextFrame getFrame frames = do
  frame <- getFrame
  case frame of 
    Nothing -> do 
                 printStatus "No more frames found." "Video"
                 return frames
    _       -> do                             
                 newFrameData <- fmap ImageRGB8 . swap . fromJust <$> getFrame 
                 printStatus ("Frame: " ++ (show $ length frames) ++ " added.") "Video"
                 addNextFrame getFrame (frames ++ [newFrameData]) 

Where I am stuck / The code that should convert images to video:

-- Converts from several images to video
juicyToFFmpeg :: [Image PixelYA8] -> ?
juicyToFFmpeg imgs = undefined

Solution

  • Disclaimer: not familiar with these libraries. Information gleaned from type signatures and documentation.

    Use Codec.FFmpeg.Juicy:

    Codec.FFmpeg.Juicy.imageWriter ::
      JuicyPixelFormat p =>
      EncodingParams ->
      FilePath ->
      IO (
        Maybe (Image p) -> 
        IO ()
      )
    

    Define

    instance JuicyPixelFormat PixelYA8 where
      juicyPixelFormat _  = _ -- in memory format of PixelYA8
    
    juicyToFFmpeg :: [Image PixelYA8] -> FilePath -> IO ()
    juicyToFFmpeg is fp = do writer <- imageWriter params fp
                             -- give Just image data to writer to append it
                             forM_ is (writer . Just)
                             writer Nothing -- finalize, or else you'll break it
       where params :: EncodingParams
             params = _ -- Figure out what fps, width, height, etc. you want (hardcode? parameters to juicyToFFmpeg? fold on is?)
    

    Old Answer

    juicyToFFmpeg :: [Image PixelYA8] -> FilePath -> IO ()
    juicyToFFmpeg is fp = do writer <- frameWriter (findEncodingParams is) fp
                             forM_ is $ \img -> let form = imgForm img
                                                    dims = imgDims img
                                                    pixs = imgData img
                                                in writer $ Just (form, dims, pixs)
                             writer Nothing
       where findEncodingParams = _
             imgForm :: Image PixelYA8 -> AVPixelFormat
             imgForm = _ -- however your images are encoded
             imgDims :: Image PixelYA8 -> V2 Int
             imgDims = _ -- duh
             imgData :: Image PixelYA8 -> Vector CUInt
             imgData = _ -- Encode as a bunch of integers, following the value of imgForm