Atypical Uses for Makefiles

Makefiles traditionally center themselves in a build process.  While they have taken a backseat to more modern build utilities, oftentimes they are simply that....in the backseat.  Eclipse, for example, autogenerated makefiles as part of a project build.  Why?  Because when properly done, makefiles are extraordinarily powerful.  Its dependency engine can parse large projects and selectively execute only what needs to be rebuilt.  Done poorly, you're at the mercy of 'make clean; make all' sequences to make sure everything is built up-to-date.

Aside from it's power as a build utility, make can be useful for other purposes as well.  Over the past couple years I've extended my use of make into a few additional areas where I find the dependency engine to prove to be very useful.  This post will touch on a few.

Quick Introduction to Makefiles

Whether you are familiar with makefiles or not, consider taking a peek at this past post.  While it touches on the general syntax and utility of make, in the later sections it shows some atypical uses.  As a means to demonstrate the dependency engine, it shows how Imagemagick can be used to transform images of varying file types.  A daisy-chained dependency chain can be created by converting an image through a series of steps JPG => PNG => GIF => JP2 => XWD.  Perhaps unpractical, but it shows an atypical use of make and it provides a simple and visible execution of the dependency engine.

Poor Man's Parallelism

I'm particularly proud of this usage, to my knowledge no one else has ever recommended this as a use of make.  Most, familiar with make, know that it can be multi-threaded and the dependency engine fires off the specified number of threads to speed up the process.  Given a large text file, make can provide an easy means to parallelize the process.  The trick; split the file into segments, process each segment, then join the results.  In ~30 lines of code, you can greatly improve the execution time for simple parallel processing, all enabled by the sophisticated dependency engine at the core of make.  Take a peek at the details here.

FFmpeg

While I've toyed to a good degree with bash scripts, python scripts and a variety of other shell utilities I've come home to make as the best tool for FFmpeg-based tasks.  Video conversions and/or modifications lend themselves nicely to makefiles.  Suppose you have a large directory full of images and you wish to create a slideshow....this post is a good starting point to showing you how makefiles can make that an easy process.  Want to take 6 hours of raw camera footage and transform it into something worth watchable....this post shows how that can be done.  Want to download a Youtube video and apply a series of transformations to create some completely new content.....this post can give you a head start.

Some of the beauties that were created by these posts;

Shell Script Replacements

Makefiles have become my adhoc replacements for simple shell scripts.  Certainly, they are limited to execute simplistic series of commands and are no substitute for sophisticated needs, but chances are if you have to execute a series of shell commands in a pre-defined order a makefile will scratch that itch.  Sophisticated recipes can become clumsy, and I've found myself authoring some ugly, ugly, ugly recipes, but recently I found a workaround.  Suppose you have a series of 5 tightly-bound commands and find it difficult to define as a recipe, a trick I've found useful is creating a shell script by means of a recipe, then using it in another target.

all: somethingCool.sh

        ${SH} ./$<

somethingCool.sh:

        ${SH} echo "echo 'doing something cool'" > $@

        ${SH} echo "sleep 1" >> $@

        ${SH} chmod a+x $@

clean:

        ${RM} somethingCool.sh

Notice the helper shell script is created by make as a dependency of the all target, used as needed and can be safely removed, recreated when later needed.  Certainly, this falls apart if the creation of the shell script by make is unnecessarily complex, but in many of my projects the only file under version control is the makefile, the rest are created as part of the project and removed by the clean target.  Nice and clean.  Rapid prototyping an alternative script can be done by a new target and once you get the hang of it can prove to be quite powerful.  The make, make run, make clean becomes the holy trinity of rapid prototyping.

I've found adhoc usage reports, grabbing debug logs, grepping for significant events, sorting and tallying event counts really align with raw, events, report targets and recipes.