In the ongoing saga of What doesn't work after the upgrade? I tried to run my logo-overlay script the other day to add the Linux Journal logo to a video and, as you may have guessed, it didn't work. In this case I'd been forewarned...

When I say I'd been forewarned, what I mean is that a reader/commentor on one of my posts had alerted me to the fact that the feature in ffmpeg that I was using (the "vhook" feature) was being deprecated and had already been removed from the development trunk. Of course I didn't do anything about it at the time, cuz hey, it's working for me. Well, now it's not!

The latest versions of ffmpeg have replaced the "vhook" mechanism with "avfilters". These are not the actual overlay feature, rather they are the way that "filters" were introduced into the processing chain. The new "avfilters" supposedly contain an "overlay" filter, but the avfilter capability does not appear to be compiled into the openSUSE version of ffmpeg. I downloaded the latest ffmpeg from development repository and configured it and built it with the avfilters enabled but that didn't produce the overlay filter either. Furthermore, I couldn't find anything in the downloaded source that appeared to be an overlay filter. I didn't feel like spending too much time trying to figure that out so I started looking for another solution.

I started looking at mencoder (part of the mplayer project) and noticed the -vf bmovl option, which is the bitmap overlay (bmovl) video filter (-vf). My first attempts with it didn't seem to be working but fortunately someone else had already figured it out and documented it here. I thought it would be useful to reproduce the guts of it here and add a bit of my own explanation as well as show a bit more of what can be done with this feature. The following is in essence the same as what you'll find at the link above, it overlays a bitmap onto a video and outputs a new video.

logo=logo.png
logo_width=600
logo_height=500
input_video=input.ogv
output_video=output.raw

rm -f $output_video
rm -f tfifo
rm -f tlogo.rgba
mkfifo tfifo

# Convert logo to RGBA.
convert $logo tlogo.rgba

# Copy logo to fifo.
(echo "RGBA32 $logo_width $logo_height 0 0 0 1" ; cat tlogo.rgba ) >tfifo &

# Convert input video with overlay from fifo.
mencoder -oac pcm -ovc raw -vf bmovl=0:0:tfifo -o $output_video $input_video

rm -f tfifo
rm -f tlogo.rgba

Mencoder does video encoding/converting much the same as ffmpeg does. One of its features is the ability to overlay data coming from a fifo on top of the video. In this case I'm taking the logo that I want to overlay and converting it to RGBA32 format (which is essentially just the uncompressed/unencode image data, 32 bits for each pixel). Then a mencoder fifo command and the RGBA32 data are fed into the fifo:

(echo "RGBA32 $logo_width $logo_height 0 0 0 1" ; cat tlogo.rgba ) >tfifo &

Mencoder runs in the foreground and converts the input video while simultaneously overlaying the data coming from the fifo.

mencoder -oac pcm -ovc raw -vf bmovl=0:0:tfifo -o $output_video $input_video

Note that for this to be of any value your logo (overlay bitmap) needs to have an alpha channel, i.e. it needs to be transparent everywhere except where the logo is.

The option of interest here is the -vf bmovl=0:0:tfifo option. This tells mencoder to apply a video filter (-vf) to the conversion/encoding, in this case the bitmap overlay filter (bmovl). The bmovl options are: hidden:opaque:fifo, where:

• hidden specifies whether the overlaid bitmap should hidden or visible, here 0 means visible (not hidden).
• opaque specifies whether the overlaid bitmap should allow the video to show through or should block it, here 0 means transparent (not opaque) so that the transparent part of the logo bitmap does not block the rest of the video.
• fifo is just the name of the fifo file, here it is tfifo.

Also of interest is the fifo command that precedes the bitmap data in the fifo: RGBA32 $logo_width $logo_height 0 0 0 1. The RGBA32 specifies that the data is in RGBA32 format, its parameters are "width height xpos ypos alpha clear", where:

• width / height specify the width and height of the bitmap data in pixels (not bytes). The bitmap file (the .rgba file) should be width * height * 4 bytes in size. Here the size of the logo overlay image is specified.
• xpos / ypos specify the X and Y position in the video at which the bitmap data is to be overlaid. Here the position is simply 0,0.
• alpha allows you to modify the alpha-ness of your bitmap (see below). Here 0 means don't change the alpha-ness of the bitmap.
• clear specifies if the existing bitmap data in the fifo is cleared before applying the new bitmap data or if the new bitmap data is overwritten on top of the existing bitmap data. Here 1 means clear the data, this may be moot since we only have one bitmap image but just in case there's garbage by default we specify 1 to clear it.

The alpha option probably requires a bit more explanation. The alpha option takes a value between -255 and 255. Values from -1 down to -255 make your bitmap ever more transparent allowing more of the video to show through, at -255 your bitmap is completely transparent and only the video will be visible. Values from 1 up to 255 make your bitmap ever more opaque allowing less of the video to show through, at 255 your bitmap is completely opaque and none of the video will be visibile in the area where the bitmap is.

To back up a bit, the mencoder/mplayer fifo is not a single command fifo, it can accept multiple commands and bitmaps (if the command requires bitmap data). So by passing increasing or decreasing alpha values we can fade in and fade out bitmap overlays on a video. For example, we can fade in and then fade out a bitmap on top of a playing video with the following sequence of commands:

rm -f tfifo
rm -f overlay.rgba
mkfifo tfifo
convert overlay.png overlay.rgba

(
	alpha=-255
	fade=20
	while [[ $alpha -le 0 ]]
	do
		(echo "RGBA32 200 200 0 0 $alpha 1"; cat overlay.rgba) >tfifo
		sleep 0.1
		let t=alpha+fade
		if [[ $alpha -ne 0  &&  $t -gt 0 ]]; then t=0; fi
		alpha=$t
	done
	sleep 0.5
	while [[ $alpha -ge -255 ]]
	do
		(echo "RGBA32 200 200 0 0 $alpha 1"; cat overlay.rgba) >tfifo
		sleep 0.1
		let t=alpha-fade
		if [[ $alpha -ne 0  &&  $t -le -255 ]]; then t=-255; fi
		alpha=$t
	done
) &
pid=$!

mplayer -vf bmovl=0:0:tfifo video.ogv

rm -f tfifo
rm -f overlay.rgba
if kill -0 $pid; then kill -9 $pid; fi

You'll notice that this script uses mplayer and not mencoder. My initial thought was that I could do this with mencoder to overlay the sequence of bitmaps as the video was encoded by mencoder and save the video in a new file, but so far I haven't been able to make that happen. Mencoder just encodes and so it doesn't sequence things at the rate that data appears in the fifo: it just goes as fast as it can, which means most of the fifo commands are missed. There may be a way to make it work, but I haven't found it yet. See the attached video for a sample of run of the script.

Back to the original problem though for just a second, that of overlaying the LJ logo on a video. Using mencoder turned out to be a less than useful solution since in addition to my overlay script breaking, my video recorder solution (xvidcap) also is broken after the upgrade. It captures video but it's jerky on playback. So that required me to change recorder programs and start using recordmydesktop for recording video. It works reasonably well (it did not the last time I'd tried it) although xvidcap has a better UI.

However, recordmydesktop only captures .ogv video and mencoder won't output .ogv files. So I had to overlay the logo onto the captured .ogv file and output raw video. Then I had to use ffmpeg to convert the raw video (with the logo) back to .ogv (and to .mp4) so that I had both .ogv and .mp4 files. Unfortunately, ffmpeg screws up the audio/video sync in the converted .ogv file, so using that sequence of operations wasn't going to work very well.

My final solution will wait till next time. And my apologies for this twisted tail.