FFMpeg Audio Waveform

If you've spent any time on this blog you've likely seen many instances of using Ffmpeg on video.  Ffmpeg also provides functionality for audio and this post will focus on generating a graphical waveform from the audio and overlaying it alongside the video.

For this, we need Johnny...Johnny Cash.

What we're gonna do here is: download the video, extend the video height to accommodate a graphical waveform on the bottom, generate a waveform video and overlay it along the bottom of the output video.

This is our bogey;

Let's download the video;
$ youtube-dl https://www.youtube.com/watch?v=8AHCfZTRGiI

Next, generate the audio waveform from the video, note the same width of the input video and a fixed height of 120;
$ ffmpeg -i input.mp4 -filter_complex "[0:a]showwaves=s=854x120:mode=line,format=yuv420p[v]" -map "[v]" -map 0:a -c:v libx264 -c:a copy waveform.mp4

Lastly, overlay the waveform on the bottom of the frame after padding the bottom to account for the additional video space, note the use of the waveform video height (e.g. 120);

$ ffmpeg -i input.mp4 -vf 'pad=iw:ih+120 [top]; movie=waveform.mp4 [bottom];[top][bottom] overlay=0:main_h-120' -acodec copy output.mp4

You're left with Johnny and his amazing vocal range co-star'ing;

Now go make something cool.

Formal Names For Common C Techniques

If you ever written a C application with custom-defined header files, you've used the pre-processor pattern:


#ifndef FOO_H
#define FOO_H
.
.
.
#endif


Ever wonder what this pattern was called? I never did, until the other day when I searched for it. The pattern is referred to as the 'once-only header' pattern, sometimes called the 'wrapper #ifdef'.

Call it what you want, the intention is to ensure that the header is not copy-n-pasted multiple times in a translation unit, due to multiple direct or indirect includes. The FOO_H is referred to as the 'guard' or 'controlling macro'.

Another common technique that you may have made use of goes something like this:

#ifdef SYS1
#include "sys1.h"
#elif . . .
.
.
.
#endif

This is referred to as a 'computed includes' pattern or technique. Not much else to add, just a name for a common face.

Displaying ImageMagick Output Direct To Display

One slight inconvenience to image processing is that it typically is a two-step process: 1) convert an image to an output image, followed by 2) displaying the output image.  Then, you repeat these steps until you reach your desired effect.

Not really a big inconvenience, but you can avoid the file storage and simply display the output image directly to the screen.  Let's look at an example.

$ convert Downloads/boink.jpg -resize 120% /tmp/foo.jpg
$ display /tmp/foo.jpg

The above commands will grow the input image by 20%.

You can however skip the intermediate file and display the results directly to the screen.  This is done by specifying the '-' as the output variable, which represents stdin/stdout.  In other words, the following command says to increase the input file by 20% and write the results directly to stdout.

$ convert Downloads/boink.jpg -resize 120% -

But, unless you are fluent in jpg it looks like gobbly-gook, but piping it to something that speaks jpg will suite out needs.  Since display is used to display image files, it certainly can render the image.  It also allows '-' as a file designator, but in this case 'display -' means display the contents of stdin.

So, chaining the convert (writing to stdout) with the display (reading from stdin) will pipe the output file directly to display, which in turn renders it to the screen.

$ convert Downloads/boink.jpg -resize 120% - | display -

Easy Peazy.

Global Variables in C

First the disclaimers;
Global variables are bad. Global variables are evil. Global variables are, and will continue to be much like a 6-ft chainsaw, extremely dangerous to use, can regularly be replaced with a safer tool/technique, but are sometimes the proper tool for the job (like cutting down 12-ft diameter trees). None-the-less, on the rare occasion that a global variable is the proper technique, you'll need to know how to define and use them...the subject of the remainder of this post.

The following example will be used to explain techniques for defining and using globals, we will speak to it for the remainder of the post.

package1.h

1 #ifndef PACKAGE1_H
2 #define PACKAGE1_H
3 static const float GlobFloat = 3.14159;
4 static double GlobDouble = 1.23456;
5 extern unsigned GlobUnsigned;
6 //char GlobChar='a';
7
8 void func1(void);
9 #endif

package1.c

1 #include "package1.h"
2 #include
3
4 static int PackageGlobalInt = 1;
5 unsigned GlobUnsigned = 12345;
6
7 void func1() {
8 printf("(%s:%d) PackageGlobalInt(%d)\n",__FILE__,__LINE__,PackageGlobalInt++);
9 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
10 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
11 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
12 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
13 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
14 }
15

package2.h

1 #ifndef PACKAGE2_H
2 #define PACKAGE2_H
3 #include "package1.h"
4
5 extern long GlobLong;
6 void func2(void);
7 #endif

package2.c

1 #include "package2.h"
2 #include
3
4 static int PackageGlobalInt = 2;
5 long GlobLong = 987654321;
6
7 void func2() {
8 printf("(%s:%d) PackageGlobalInt(%d)\n",__FILE__,__LINE__,PackageGlobalInt++);
9 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
10 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
11 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
12 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong++);
13 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble++);
14 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned++);
15 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong++);
16 }
17

main.c

1 #include
2 #include "package1.h"
3 #include "package2.h"
4
5 int main() {
6 printf("(%s:%d) main process initializing\n",__FILE__,__LINE__);
7
8 func1();
9 func2();
10
11 printf("(%s:%d) GlobFloat(%f)\n",__FILE__,__LINE__,GlobFloat);
12 printf("(%s:%d) GlobDouble(%lf)\n",__FILE__,__LINE__,GlobDouble);
13 printf("(%s:%d) GlobUnsigned(%u)\n",__FILE__,__LINE__,GlobUnsigned);
14 printf("(%s:%d) GlobLong(%li)\n",__FILE__,__LINE__,GlobLong);
15
16
17 printf("(%s:%d) main process terminating\n",__FILE__,__LINE__);
18 }
19


Globals are a form of interface, so rightly are typically defined in a header file. Package 1 attempts to define 3 global variables. You will soon see that it was only successful in defining one. Before we get into the details, we should review some preliminaries.

#includes

As you should already be aware, header files serve little purpose other than organization of code. When encountering a #include directive, the precompiler does little more than copy-n-pasting the contents into the sourcing file. This is significant in our later discussion.

Static Variables

Static variables cannot be accessed outside a translation unit. For all intents and purposes, a translation unit is a C source file. This, along with what we know about #includes will explain why only one of our intended global variable declarations was successful.

Back To Our Example

The contents of package1.h on lines 3-5 are three attempts to declare a global variable. Note, lines 3-4 define the variables as static, line 5 defining it as external linkage. Why are the first two variables defined as static??? Because if I don't, I get a linker error "multiple definition of . . ." error. Pounding on the keyboard, slowly beating the linker into submission is accomplished by declaring the variable as static. Finally, the linker shuts up; we must have fixed it, right? Wrong!

So, why doesn't lines 3-4 work? More importantly, how do we know it doesn't work? Well, we can confirm that line 4 didn't work by comparing the results of lines package1.c:10, package1.c:12, package2.c:10, package2.c:13, and main.c:12. Note that on package 1 & 2 functions, we are incrementing the contents of GlobDouble. If we were successful in defining a global variable, we should see the variable initially assigned 1.23456, incremented 4 times with a final result of 5.23456.


~/StorageClasses$ ./main | grep GlobDouble
(package1.c:10) GlobDouble(1.234560)
(package1.c:12) GlobDouble(2.234560)
(package2.c:10) GlobDouble(1.234560)
(package2.c:13) GlobDouble(2.234560)
(main.c:12) GlobDouble(1.234560)

Hmmmph, we didn't accomplish what we intended, did we? The static storage class, coupled with the precompiler copy-n-paste behavior resulted in 3 independent variable declarations; one in package1.c, one in package2.c, and one in main.c. Each instance is unique to each translation unit, so incrementing one only affects the declaration in that translation unit.

Line 5 however successfully defined a global variable. How? Well, line 5 simply states that we are declaring a variable, to be defined elsewhere. Declaration simply defines a reference to variable that will be defined elsewhere (in this case, line 5 in package1.c). Translation units: package1.c, package2.c and main.c will each declare a reference to GlobUnsigned, but the single definition of the variable resides in package1.c.

We can confirm that all is well by means of:


~/StorageClasses$ ./main | grep GlobUnsigned
(package1.c:11) GlobUnsigned(12345)
(package1.c:13) GlobUnsigned(12346)
(package2.c:11) GlobUnsigned(12347)
(package2.c:14) GlobUnsigned(12348)
(main.c:13) GlobUnsigned(12349)



It is worth noting that lines 3-4 in package1.h have identical results as line 4 in both package1.c and package2.c. All instances are similar in the matter that their scope is limited to the translation unit in which they are declared.