FFMpeg Dynamic Splatter Effect

In past posts we've demonstrated overlays and masks that can result in some pretty cool effects.  I vaguely recall a dynamic splatter effect where a photo (or video) was masked by a fluid mask and thought I'd spend some time trying to replicate it.  Here's what I found.

Let's spoil the reveal and jump to what we are trying to accomplish;

First step is to generate or steal the makings of a dynamic mask.  Water or fire seems a good fit, so browsed around YouTube to find something we could work with and this one caught my eye;

We can transform this splash effect upon a white background by performing pixel thresholding like this;

$ ffmpeg -i splash.mp4 -filter_complex "format=gray,geq=lum_expr='if(lte(lum(X,Y),128),255,0)'" -an -t 50 mask.mp4

Resulting in this:

Let's create our background image that we'll apply the mask to;

$ ffmpeg -loop 1 -t 50 -i image01.jpg -y image01.mp4

In our final step we'll apply the mask.  Since the mask region is an RGB video with white (255) pixels representing visible regions and black (0) regions indicating opaque we can and the video pixels to get the final result.

$ ffmpeg -i image01.mp4 -i mask.mp4 -filter_complex "[1:0] setdar=dar=1,format=rgba [a]; [0:0]setdar=dar=1,format=rgba [b]; [b][a]blend=all_mode='and'" -y clip.mp4

Apply some audio, repeat the video to fit to the audio length and you've got the resultant video.

Cheers.