ffmpegsoxnoise-reductionobs

Using an actual audio recording to filter out noise from a video


I use my laptop (Ubuntu 18.04 LTS derivative on a Dell XPS13) for recording videos (these are just narrated presentations) using OBS. After a presentation is done (.flv format), I process it using ffmpeg using filters that try to reduce background noise, reduce the size of the video, change encoding to .mp4, insert a watermark, etc. Over several months, this system has worked well.

However, my laptop is now beginning to show its age (it is 4 years old). That means that the fan becomes loud - loud enough to notice in a recording, not loud enough to notice when you are working. So, even after filtering for low frequency in ffmpeg, there are clicking and other type of sounds that are left in the video. I am a scientist, though not an audio/video expert. So, I was thinking - is it possible for me to simply record the noise coming out of my machine when I am not presenting, and then use that recording to filter out the noise that my machine makes during the presentation?

Blanket approaches like filtering out certain ranges of the audio spectrum, etc. are unlikely to work, as the power spectrum of the noise likely has many peaks, and these are likely to extend into human voice range as well (I can hear them). Further, this is a moving target - the laptop is aging and in any case, the amount and type of noise it makes depends on the load and how long it has been on. Algorithm:

  1. Record actual computer noise (with the added bonus of background noise) while I am not recording. Ideally, just before starting to record the presentation. This could take the form of a 1-2 minute audio sample.
  2. Record the presentation on OBS.
  3. Use 1 as a filter to get rid of noise in 2. I imagine it would involve doing a Fourier analysis of 1, and then removing those peaks from the spectrum of 2 at each time epoch.

I have looked into sox, which is what people somewhat flippantly point you to without giving any details. I do not know how to separate out audio channels from a video and then interleave them back together (not an expert on the software here). Other than RTFM, is there any helpful advice anyone could offer? I have searched, but have not been able to find a HOWTO. I expect that that is probably the fault of my search since I refuse to believe that this is a new idea - it is a standard method used in many fields to get rid of noise, including astronomy.


Solution

  • I don't have enough reputation to comment, so here's my answer as it applies to my situation, for which I was looking for answers myself. It may or may not apply to your situation, as I record my screen using ffmpeg (which is fine for recording everything on your screen, but not a very good option for a single program/window or a part of the screen, which can be done, but I don't think there is a visual indication of which portion of the screen is being recorded).

    I came across the afftdn filter (https://ffmpeg.org/ffmpeg-filters.html#afftdn which @Gyan also mentions in a comment to OP's question) and was able to use it successfully.

    The process is probably applicable only to a live recording being made by ffmpeg - at least I can't think of a way of doing this with a prerecorded content. The procedure below works for audio only input as well, although you'll need to modify your ffmpeg command to only record audio. It works like this:

    1. Run ffmpeg command to record your audio/screen.
    2. Instruct afftdn to record the background noise and sit quietly for a moment.
    3. Instruct afftdn to stop recording the background noise and proceed with your voice commentary and the actual recording of your presentation.
    4. Discard the beginning of the recording, which only contains the recording of your background noise, from your output file.

    For Step 1. I run

    ffmpeg -f pulse -i <my_input_device> -f x11grab -s 1920x1080 -framerate 30 -i :0.0 -s 1280x720 -filter_complex afftdn=tn=enabled /home/my_user_name/Videos/my_output_file.mp4
    

    To list your input devices run

    pactl list short sources
    

    and pick a name of the input device to use - mine is alsa_input.pci-000_00_1b.0.analog-stereo. The first -s argument to the ffmpeg command above (-s 1920x1080) is my screen resolution (adjust accordingly, also you can make this smaller than your screen resolution to record only a part of your screen - combine this with the offset argument to move the recorded part from the top left quadrant of the screen), the argument -i :0.0 indicates the top left pixel of your default screen - other offsets are possible if you don't want to record the whole screen (change your input resolution accordingly if you change this offset). The second -s argument (-s 1280x780) is the output video resolution.

    Step 2: Hit c to tell ffmpeg that you're issuing a command to a filter. ffmpeg should prompt you for an entry by outputting Enter command: <target>|all <time>| -1 <command>[ <argument>]. Type:

    afftdn -1 start
    

    The filter is now recording your background noise.

    Step 3: Hit c to tell ffmpeg that you're issuing a command to a filter again and when prompted type:

    afftdn -1 stop
    

    The filter should now be filtering the background noise from your audio (in my case it took a few seconds to kick in - I suspect this may depend on the length of the noise recording - you may need to experiment a little to get a good feel for when to start your actual recording). Proceed with your recording.

    Step 4. Play back the audio/video to find the beginning time of your actual video with noise removed. Then tell ffmpeg to cut out everything before that time:

    ffmpeg -ss <duration> -i /home/my_user_name/Videos/my_output_file.mp4 /home/my_user_name/Videos/my_output_file_with_noise_removed.mp4
    

    Replace <duration> with the number of seconds to remove from the input file (you can also provide the duration in HH:mm:ss.d format, whith hours HH and decimal .d being optional), e.g.:

    ffmpeg -ss 30.5 -i /home/my_user_name/Videos/my_output_file.mp4 /home/my_user_name/Videos/my_output_file_with_noise_removed.mp4
    

    Use the file names you prefer, of course - I like to just navigate to the working directory and issue commands of the form

    ffmpeg -ss 30.5 -i input_file.mp4 noiseless_output_file.mp4