Sunday, October 16, 2016

Generating a Video Collage -- Get Your Rock On!

I began with an overall objective of being able to:
1) get a list of the top 100 songs for a given year from the Billboard Charts
2) perform a search for each song on YouTube using the title and artist
3) download the song
4) overlay the title/artist on the video
5) finally, create a video collage consisting of a 30 sec video clip for each song

Let's walk through the steps.  Let's focus on the year 1994 to pay tribute to the high school graduation year of my good friend Marshall, likely the only reader of this blog.  That shows true dedication 'Old Fashioned', poor judgement in good use of your time, but true dedication.

Step 1 -- Get Top 100 Song List

I first took the path of parsing the HTML and extracting the music list into a series of strings but I later abandoned that effort by dusting off an old tool that I last used back in college, say 1996-1998'ish.  Lynx is a text-based web browser with a sexy little secret, the ability to dump to a text file, making parsing the contents significantly easier.

$ lynx --dump -nonumbers  > list.txt
$ cat list.txt
     1   #alternate Edit this page Wikipedia (en) copyright
     3 Billboard Year-End Hot 100 singles of 1994
     5   From Wikipedia, the free encyclopedia
     6   Jump to: navigation, search
     8   This is a list of Billboard magazine's Top Hot 100 songs of 1994.^[1]
     9   № Title Artist(s)
    10   1 "The Sign" Ace of Base
    11   2 "I Swear" All-4-One
    12   3 "I'll Make Love to You" Boyz II Men
    13   4 "The Power of Love" CĂ©line Dion
    14   5 "Hero" Mariah Carey
    15   6 "Stay (I Missed You)" Lisa Loeb and Nine Stories
    16   7 "Breathe Again" Toni Braxton
    17   8 "All for Love" Bryan Adams, Rod Stewart and Sting
    18   9 "All That She Wants" Ace of Base
    19   10 "Don't Turn Around" Ace of Base
    20   11 "Bump n' Grind" R. Kelly
    21   12 "Again" Janet Jackson

This gives us something we can easily work with.  If we grab the title list using awk:
$ awk '/Title/,/ 100 /' ./list.txt 
   № Title Artist(s)
   1 "The Sign" Ace of Base
   2 "I Swear" All-4-One
   3 "I'll Make Love to You" Boyz II Men
   97 "What Is Love" Haddaway
   98 "And Our Feelings" Babyface
   99 "Bop Gun (One Nation)" Ice Cube featuring George Clinton
   100 "I Wanna Be Down" Brandy

For each of these titles, snag the artist and title and prepare to issue a respective Youtube-dl command.

Step 2-3 -- Search and Download Source Videos

Once we have a title and artist we should have sufficient info to perform a search, by issuing commands resembling the following:

 $ youtube-dl --verbose "gvsearch1:the sign ace if base" -f 'bestvideo[ext=mp4]+bestaudio[ext=m4a]/best[ext=mp4]/best' --output Yt-001

This command will perform a Youtube search and prioritize videos of MP4 format.  The output specifies a prefix, there is not guarantee as to the format.  While there are pages of documentation on Youtube-dl we won't get into detail here, know however you can tailor your search to limit format, quality, resolution. frame-rate,....  Given we aren't constraining such video qualities we'll enforce some constraints post-processing the videos after we've downloaded them.  We can ensure a H264 format within an MP4 container by examining what was downloaded and reencode it if necessary;

    if [ ! -e $prefixName.mp4 ]; then
      ffmpeg -y -i $prefixName* -strict -2 -f h264 $prefixName.mp4

This is a good stage to generate a PNG image to overlay that consists of the List #, Artist and Title:
$ convert -background white -fill black -font FreeSerif-Italic -pointsize 46 label:"1 - Ace of Base - The Sign" 001.png

Step 4 -- Overlay the Title/Artist on the Video

Ensuring the videos share consist resolution and frame-rate will make our lives much easier.  For instance, the the font size of overlay PNG is aimed at 720p videos.  When we later concatenate the videos into a single video you'll find the input videos need to share a consistent frame-rate or the audio falls out of sync and video can stall out.  Let's transcode the Youtube video, normalize it to 720p, 30fps and let's seek in 60 seconds and grab a 30 sec clip.
$ ffmpeg -y -i Yt-001.mp4 -vf "[in] scale=hd720 [top]; movie=001.png[bottom]; [top][bottom] overlay=0:H-56 [out]" -strict -2 -ss 60 -t 15 -r 30 clip-001.mp4

For the observant reader, you may notice the overlay position of (H-56), this places the PNG image at x,y location 0,720-56.  For purposes of understanding, I provided the height dimension of the PNG image which places the overlay where we want it.

Step 5 -- Concatenate the Videos

The last step was the one that gave me the most trouble and made me miss my self-enforced blog deadline last week.  Concatenating the videos occasionally resulted in stalled video or Kung Fu Theatre style audio out-of-sync issues.  Despite many a Google search, I finally found through extensive debugging that the cause of the problem was supplying input videos that didn't have a consistent frame-rate so take care to enforce that before attempting this step like we did in the previous step.

Concating the videos is pretty straightforward: 1) provide a list of video files in a text file and 2) issue a concat filter via FFMpeg.

$ cat files.txt 
file 'clip-001.mp4'
file 'clip-002.mp4'
file 'clip-003.mp4'
file 'clip-004.mp4'

The text file (e.g. files.txt) needs to be located in the same working directory where you issue the FFMpeg command.  

  $ ffmpeg -y -f concat -i files.txt -strict -2 -r 30 video.mp4

A crudely authored script is available that automates this process.  Feel free to take a peek;

The output; the top 100 songs off the Billboard Chart for 1994:

Rock On 'Old Fashioned'!

No comments:

Post a Comment