Saturday, August 31, 2019

Cartoonize Video Using FFMpeg and Imagemagick

The year would be 1985, Raybans and Rubiks cubes were all the rage and a iconic MTV video hit stormed the MTV circuit.
This one-hit-wonder holds a special place in my heart.  The black-n-white sketch effect to this day is pretty spectacular, thought I'd take a crack at cartoonizing a video using FFMpeg and Imagemagick.  I've yet to get an equivalent effect, but it's still a fun exercise.

Let's use the amazing movie remake 'A Man On Fire' as our source video.

 Let's say we have a scene like this.



And we wish to make it look like this.


Doing this for each frame of the video should give us a sketch or cartoon effect, like a comic book effect.  A far cry from 'Take On Me' but perhaps a step in the right path.

While FFMpeg has performed miracles in the past, we'll have to pair it up with ImageMagick to get this effect.  In short, we'll extract the video into a series of png frames, run each frame through ImageMagick to cartoonize the frame, then we'll reassemble the images into a video with FFMpeg once again.

Say we have an input video file: ManOnFire.mpg

Let's grab a clip;
$ ffmpeg -i ManOnFire.mpg -ss 373 -t 3 -strict -2 clip01.mp4

Then let's extract it into a series of frames;

$ ffmpeg -y -i clip01.mp4 -an -vf scale=360x240 clip01.mp4-f%05d.png -strict -2 clip01.mp4-audio.mp3

Then, let's convert each frame into a cartoonized frame; grab a sandwich and a beer, this step is pretty time-intensive.

$ find . -name "clip01.mp4-*.png" -exec convert {} -sketch 0x20+135 {} \;

Re-add the audio:
$ ffmpeg -y -r 30000/1001 -i clip01.mp4-f%05d.png -i clip01.mp4-audio.mp3 -c:a copy cartoon01.mp4


Do this for a series of clips and you've got a cartoon highlight real of an exceptional movie.

Yes, it's true; Cartoon Denzel has as much swagger as real-life Denzel.

Sunday, August 25, 2019

My Mentor -- Pam Ogaard Padgett



Let me introduce you to a remarkable lady, Pam Ogaard Padgett.

Mrs. O, how we tended to refer to her in years past, was my first college professor and directly responsible for me pursuing Computer Science as a career.  I owe much to this lady: a challenging and fulfilling career, a recognition of what makes a great teacher, a respect for a love for mathematics, and indirectly the relationship to the woman I married, the car I drive and the freedoms that of the engineering lifestyle I live.

Three women are responsible for taking an immature, unfocused, impulsive young adult and largely molding him into the man he is today.  The first two; my mother and Pam, got me on the right track.  The third, my wife, accepted the tiresome task years later and continues to work on me to this day.

Throughout high school I was a terrible student.  A 'last minute Charlie' when it came to homework, test prep, and pretty much anything academic.  I'm convinced that my high school teachers took into consideration that if they didn't pass me, they'd be penalized by having me yet another year. 🙂

While I liked computers from an early'ish age, I don't think I considered it a career or profession 'til much later in my life.  I felt college was my course in life, not exactly knowing why, and I was certainly conflicted; "How could a C-student who didn't tend to apply himself ever graduate college?".  But, college was the plan, and the summer was focused on paying for it.

I worked heavy equipment construction for the summer under the supervision of my father.  Long days of hard work, academia was furthest from my (or anyone else's) mind.  Twelve hour days + commuting time left little time to conduct deep thought into life choices or career path.  But, the summer was coming to an end, and the fall semester was approaching and little Grant had absolutely no idea what he wanted to be when he grew up.  I liked computers, but I liked all kinds of things, so I didn't feel particularly drawn to them.

Fall is coming, applications were accepted, course selection looming.  A college course work booklet sat on the nightstand like many textbooks of the past.  You know you should crack it open and make a decision, but years of hard-wired procrastination tendencies took hold.  Pair that with not knowing what I wanted to be certainly complicated matters.

This is when Pam and my mother set the course for the rest of my life.  Such an influential moment of a man's life should be painted in words vibrantly and with detail.  "It was a Thursday evening on a fleeting summer,..." a more talented story teller may spin such a yarn, but the years and/or mileage on the old brain prevent recalling in detail and I'm not much for fibbing on details.

Still working long days meant leaving at 5:30-6 am and returning around 7 pm typically.  Any career counseling or questions during typical business hours were impractical.  It would be decades before the advent of cell phones and while the field office had a phone there was little time for personal calls and frankly they were frowned upon.  So, my mother took the reigns a bit.  "You like computers, don't you"; asked one evening, "yes".  Brief, directed, and short; not really a significant discussion and I honestly thought nothing of it at the time.  If life were produced in the manner of Hollywood films, at this moment the lighting would have slightly intensified, a slight soundtrack would be played and dramatic pauses would have been scripted; in a manner of saying 'Something important happened here'.

The following evening I came home, covered in dirt, oil, grease and an odor resembling the stink of a mastodon.  I hustled into the shower and headed upstairs to grab dinner, my mother handed me a slip of paper afterwards with a name and telephone number.  "She's expecting your call" my mother instructed.  This was my first encounter with Mrs O.  Someone who was willing to give out her personal phone number, giving up a chunk of her evening to talk to someone she had never met simply to provide some guidance to a would-be student.  I remember little of the conversation, vaguely asking what I'd refer to as 'dumb questions' at the time.  I didn't know how computers could become a career or whether I'd be a fit for them.  This was a pre-geek-chiche culture, the television had Magnum PI on it, not Numbers.  Think 'Revenge of the Nerds' rather than Abby from 'NCIS'.  The computer industry was still forming, the field wasn't as full-formed as it is today.  Hobbyists with computers in their garage for fun and NASA, a kid like me didn't see many opportunities in between.  Mrs O filled in some of the gaps, patiently and clearly.  At the end of the conversation I left with a course list and a mild interest in 'giving it a try'.

I first began as her student, taking introductory computer classes.  Programming in PASCAL, later MODULA-2 followed by assembler....the language any computer god-fearing mortal fears and respects.  Mrs O was and incredible instructor, a dedicated teacher and a patient presenter.  Classes filled with high-strung young adults who could likely have benefited from a Ritalin-dipped veterinarian prescribed darting.  Mrs O's classes were structured, planned and insightful.  While I tolerated other classes we sincerely looked forward to Mrs O's.  We learned what it meant to be a mathematician, an engineer, and honestly a better student.  Young adults are horrifying students, my opinion anyway, and I have considered writing formal apologies to many of my past instructors.  Best case, I'm bored and distracted, worst case I'm bored and disruptive.  College is the place you are when you begin growing into the adult you wish to become.  Attention-grabbing smart-ass remarks were met with patience and redirection back to the lesson.  Mrs O was a talented and experienced instructor, the way she handled our immature hijinx both defused and taught; not just the lesson at hand, but how adults should behave.

As the year went on, a true mentorship evolved.  Between classes we'd drop by Mrs O's office and chat, sometimes about the coursework, sometimes just social.  Her computer lab became our computer lab, we began taking pride in it and took responsibility for keeping it in shape; custodial, security and socially.  We became 'lab rats' in a form, doing our work as well as seeking out new challenges.  We helped underclass students when we could, mostly modeling the give-forward attitude that Mrs O lead by.  In time, Mrs O taught me to take the 'hard work' characteristic that I portrayed in physical labor into the academic form.  I worked harder than I had ever in high school, became more organized, more focused.  Without that, a 110% probability that I wouldn't have completed college.  Mrs O lead by example, always prepared for the day, always working hard.

At BSC, a 2-year college, Mrs O's goal was typically to set the stage for achieving an associates degree with the intent of transferring on to a 4-year university for continued education.  A bazillion questions from a bazillion students, Mrs O provided guidance as encouragement.  She coordinated visiting UND and NDSU with the potential transfer students.  On the visit, she met with university faculty to ensure BSC curriculum continued to satisfy the transfer university needs.  The trip was productive, fun and essential in helping transitioning to the next step in academia.  Mrs O made time to organize as well as participate.  She made the time for it, recognized a need and made it happen.  Review as many job descriptions as you wish for college professors, taking your students on a field trip to ease the decision and transition isn't anywhere on them.  That's above and beyond.

At UND, 3 hours away, a continued hectic workload, I only infrequently seen my mentor.  At the Kirkwood Mall on one, or so, occasions.  Life gets in the way.  Mileposts grow between you and time is lost.  If you're lucky, you take the time to reflect how you've gotten to where you are and recognize the people that were instrumental in getting there.  A better person would give thanks frequently and in-person and perhaps that day will come.

The last memory I will share, in close, concerns Richard Dreyfuss.  Mrs O and a number of us went to "Mr. Holland's Opus", a pretty great movie, and when we left the theater Mrs O (who primarily considers herself a math teacher I infer) said "How does a math teacher get an opus?".  Whelp, speaking for some of your students, who couldn't carry a tune in a bucket nor clap at a steady rate; we can only give you our eternal thanks, introverts by nature, our success is your success, our accomplishments are your accomplishments, our love for the profession was inspired by your love for the profession.

Thank you for setting me on this never-ending, ever-loving journey that is 'Computer Science'.





Sunday, August 18, 2019

Real-Time Data Analysis From Debug Logs


Debug logs contain a plethora of business and system information.  Most commonly, the logs are cached and post processed, but occasionally it's valuable to process the logs as they are being appended to.  For instance, monitoring the current system state from logs requires a means to read the logs as they are being updated, extract log events and calculating system metrics from the events.

Let's use a simple example, one that monitors (or follows) a 'top.dat' file which is generated by piping 'top' to the file.  The python monitoring utility will process the 'top.dat' file log events as they are written.  Since we wish to monitor the log file(s) and display metrics along the way, we need at least two threads of control: one to monitor and extract information from the 'top.dat' file, the other to display info to the user.  This example calculates the cpu load by extracting the idle percentage from top.

     1 #!/usr/bin/python
     2 import logging
     3 import threading
     4 import time
     5 import glob;
     6 import re;
     7 import datetime;
     8
     9 class Display:
    10   def __init__(self):
    11     self.lock_ = threading.Lock();
    12     self.map_=dict();
    13     pass;
    14   
    15   def update(self,key,val):
    16     self.lock_.acquire();
    17     self.map_[key]=val;
    18     self.lock_.release();
    19   
    20   def display(self):
    21     self.lock_.acquire();
    22     print "---------------"
    23     print datetime.datetime.now();
    24     for k in self.map_.keys():
    25       print "%s : %s"%(k,self.map_[k]);
    26     print "\n"
    27     self.lock_.release();
    28
    29 display=Display();
    30
    31 class fileProcessor:
    32   def __init__(self, fileName): 
    33     self.fileName_=fileName;
    34     logging.info("following %s"%(fileName));
    35     self.follow(open(self.fileName_,'r'));
    36
    37   def follow(self, fp):
    38     fp.seek(0,2);
    39     while True:
    40         line = fp.readline();
    41         if not line:
    42             time.sleep(0.1);
    43             continue;
    44         self.handle(line);
    45
    46   def handle(self, line):
    47 #   logging.info("processing line %s"%(line));
    48     m=re.match('.+, (.+) id,.*',line);
    49     if (m):
    50 #     print m.group(0);
    51       cpuLoad=100.0-float(m.group(1));
    52       print "cpuLoad: %s"%(cpuLoad);
    53       display.update('CpuLoad',cpuLoad);
    54
    55 def thread_function(name):
    56   logging.info("running %s"%(name));
    57   obj=fileProcessor(name);
    58
    59 if __name__ == "__main__":
    60   format = "%(asctime)s: %(message)s";
    61   logging.basicConfig(format=format, level=logging.INFO, datefmt="%H:%M:%S");
    62
    63   logging.info("main process initializing");
    64   t1=threading.Thread(target=thread_function, args=('./top.dat',));
    65   t1.start();
    66   logging.info("main process terminating");

Not terribly interesting, and you could readily accomplish the same thing on the main thread, there is only one file to monitor.

But suppose you wish to monitor X files, then you really need threading.

     1 #!/usr/bin/python
     2 import logging
     3 import threading
     4 import time
     5 import glob;
     6 import re;
     7 import datetime;
     8
     9 class Display:
    10   def __init__(self):
    11     self.lock_ = threading.Lock();
    12     self.map_=dict();
    13     pass;
    14   
    15   def update(self,key,val):
    16     self.lock_.acquire();
    17     self.map_[key]=val;
    18     self.lock_.release();
    19   
    20   def display(self):
    21     self.lock_.acquire();
    22     print "---------------"
    23     print datetime.datetime.now();
    24     for k in self.map_.keys():
    25       print "%s : %s"%(k,self.map_[k]);
    26     print "\n"
    27     self.lock_.release();
    28
    29 display=Display();
    30
    31 class fileProcessor:
    32   def __init__(self, fileName): 
    33     self.fileName_=fileName;
    34     logging.info("following %s"%(fileName));
    35     self.follow(open(self.fileName_,'r'));
    36
    37   def follow(self, fp):
    38     fp.seek(0,2);
    39     while True:
    40         line = fp.readline();
    41         if not line:
    42             time.sleep(0.1);
    43             continue;
    44         self.handle(line);
    45
    46   def handle(self, line):
    47 #   logging.info("processing line %s"%(line));
    48     m=re.match('.+, (.+) id,.*',line);
    49     if (m):
    50 #     print m.group(0);
    51       cpuLoad=100.0-float(m.group(1));
    52       print "cpuLoad: %s"%(cpuLoad);
    53       display.update('CpuLoad',cpuLoad);
    54
    55 def thread_function(name):
    56   logging.info("running %s"%(name));
    57   obj=fileProcessor(name);
    58
    59 if __name__ == "__main__":
    60   format = "%(asctime)s: %(message)s";
    61   logging.basicConfig(format=format, level=logging.INFO, datefmt="%H:%M:%S");
    62
    63   logging.info("main process initializing");
    64   D=dict();
    65   while(1):
    66     fList=glob.glob("./*.dat");
    67     logging.info("fList: %s"%(str(fList)));
    68      for e in fList:
    69       if not e in D.keys():
    70         logging.info("starting thread for '%s'"%(e));
    71         D[e]=threading.Thread(target=thread_function, args=(e,));
    72         D[e].start();
    73     display.display();
    74     time.sleep(3);
    75   logging.info("main process terminating");

If you had 100 '*.dat' files the above would spawn 100 threads, each following a dedicated file, each updating the display which is rendered every 3 seconds.

Just recently put something similar together to monitor dozens of user logs and tracking uptime and job submission events. Works real well, hope you have equal success.

Cheers.

Saturday, August 10, 2019

Dynamic Zoom In / Out Video Effect


This is a follow-up on our previous post: FFMpeg Zoom which demonstrated performing a dynamic zoom capability on a video.

Interest from a YouTube viewer asked how to perform zooming in, then zooming back out. I figured I'd take a crack at it.

The previous post showed how to dynamically assigning the zoompan, incrementing each video frame.  Assigning the zoompan  based on the video frame number is the key to incrementing or decrementing the zoompan which gives the effect of zooming in or out.

Let's ignore the dynamic zooming for now and focus on how to assign the zoompan based on a conditional keyed off the frame number.

If we want to zoom to 5x from 15..20 sec (assuming 30fps) in the video the following command will do just that:
$ ffmpeg -y -i target.mp4 -vf "scale=iw:ih,zoompan=z='if(between(in,450,600),5.0,1.0)':d=1:x='320-(320/zoom)':y='240-(240/zoom)'" -an zoomInOut.mp4

Replacing the static zoom coefficients (5.0 and 1.0) with dynamic calculations will give us a smooth zoom in and out effect:
$ ffmpeg -y -i target.mp4 -vf "scale=iw*4.0:ih*4.0,zoompan=z='if(between(in,0,450),min(max(zoom,pzoom)+0.050,5.0),min(max(zoom,pzoom)-0.050,5.0))':d=1:x='320.0*4.0-(320.0*4.0/zoom)':y='240.0*4.0-(240.0*4.0/zoom)'" zoomInOut.mp4

Notice the assignments are identical with exception to the sign of the arithmetic, zooming in adds 0.050, where zooming out subtracts it.

The resulting video looks like this;






Sunday, August 4, 2019

Adding Watermark To Video With FFMpeg

A pretty popular question concerning video editing is 'how do I add a watermark'? 

Overlaying a video with your logo or name is a way to get your brand out, identify your work, and get eyes on your identity.

Relatively simple to do with Ffmpeg.  You'll need as inputs a video and a suitably sized graphic.  One sweep of your FFMpeg magic wand and 'Bob's Your Uncle', pure video magic.

Lets get started:

I've referenced in past blog posts that FFMpeg and Make are a great combination for video editing, so all the steps we'll be performing can be accomplished by the following makefile:

$ cat -n Makefile 
     1 all: output.mp4
     2
     3 input.mp4 :
     4 ${SH} youtube-dl https://www.youtube.com/watch?v=Td1lyPwiM0Y -o $@
     5
     6 watermarklogo.png:
     7 ${SH} convert /mnt/svn/mal/main/trunk/FSK/Media/Graphics/logos/logo/final.gif -resize 100x100 $@
     8
     9 output.mp4: input.mp4 watermarklogo.png
    10 ${SH} ffmpeg -i $< -vf "movie=watermarklogo.png [watermark]; [in] [watermark] overlay=main_w-overlay_w-10:10 [out]" -strict -2 $@
    11
    12
    13 clean:
    14 ${RM} *.mp4 *.png

But, we'll walk through the steps individually.

Grab your video; I'm using a YouTube downloading utility to grab one of my previous videos:
$ youtube-dl https://www.youtube.com/watch?v=Td1lyPwiM0Y -o input.mp4

The video is ~47 seconds long, 640x480.


Next, find or create a 100x100 image.  I'm resizing one using ImageMagick:
$ convert final.gif -resize 100x100 watermarklogo.png



Now,  our final step is to overlay the image on the video 10 pixels from the top/right corners:

$ ffmpeg -i $< -vf "movie=watermarklogo.png [watermark]; [in] [watermark] overlay=main_w-overlay_w-10:10 [out]" -strict -2 output.mp4

abbrabracadabra


Now, go make something cool.