Sunday, April 26, 2026

Conditional Subsequent Command Calls

 

Occasionally there is a need in bash scripting to execute a series of commands in sequence conditionally.  Command2 needs to be run after command1 if, and only if, the command1 succeeded (or failed).

One means of performing this is to execute the command, assign the return value to a variable, and evaluate the return value executing the subsequent command based on the value.

 $ cat -n foo

     1  #!/bin/bash

     2

     3  sleep 1

     4  #sleep -1

     5  rc=$?

     6

     7  echo $rc

     8  if [ $rc == 0 ]; then

     9    echo "run something else"

    10  fi

    11

Note, the above the script executes a sleep command, preserves the return code, and executes the subsequent command (e.g. echo) based on the return value.  Note; we can force a failed return code by specifying a negative sleep duration.

An alternative shortened syntax is to specify the command sequence on a single line using a '&&' or '||' seperator, the seperator determing if the subsequent command should be exectued on success/failure of the preceeding command.

              command1 && command2

 This syntax issues command2 if, and only if, command1 returns an exit status of zero.

        An OR list has the form

               command1 ││ command2

Now, command2  is  executed  if  and only if command1 returns a non-zero exit status.  This can be used to readily define recovery, or notification, behaviors for a failed command. 

The return status of AND and OR lists is the exit status of the last command executed in the list.

      1  #!/bin/bash

     2

     3  sleep 1 && echo "run something else"; #--execute command2 if command1 was successful

     4  sleep -1 || echo "run something else"; #--execute command2 if command1 failed

     5  sleep -1 && echo "run something else"; #--execute command2 if command1 was successful

     6

Sunday, April 19, 2026

Read-Only Variables in Bash

 

Bash has a little known concept of read-only variables.  When specifying 'readonly' during a variable assignment prevents the variable from being reassigned.

 For example;

 1  #!/bin/bash

2

3  readonly x=0

4  echo $x

5  x=1

6  echo $x

Line 3 specifies a read-only assignment of variable x to 0, an attempt to reassign on line 5 will be disregarded and produce an error message.  The error doesn't prevent continuation of the script however.

 

::::::::::::::

Sunday, April 12, 2026

Detect Command Timeouts using Bash

 A common, and often overlooked, need for scripting is addressing hanging, or unresponsive commands.  What should your script do if one of it's ste

ps takes much, much, much longer than expected?  A hanging, or unresponsive, subprocess will result in a hanging job which in turn can prevent oth

er jobs from running.  Let's spend a bit of time how to address such an need.

 

The 'timeout' command is provided by the coreutils package, often readily available on default installation for most distributions.

 

$ sleep 10

$ echo $?

0


 

Execution of subprocesses generally set a return code, zero or non-zero, often zero indicating successful execution of the command, non-zero return codes if it failed for some reason.  The above command sequence will execute a sleep command (for 10 seconds), the second command echoing the return code.  Assignment of $? to zero indicates the sleep command executed successfully.

 

So, what if we wanted to prevent a hanging command to go undetected and/or prevent the rest of a script from running.  Let's say we want to enforce a command takes no longer than 5 seconds.
 

$ timeout 5 sleep 10

$ echo $?

124


 

After 5 seconds, the command terminates, the return code 124 indicates the command timed out.  Usage of this allows setting hard constraints on how long a command is allowed to run before terminating or being terminated.  The return code allows determining if the command timed-out.
 

Cheers.

Sunday, April 5, 2026

Synchronizing Completion of Concurrent Commands

 

Running concurrent commands in a script generally is done by executing processes in the background.  Numerous commands can be spawned concurrently then synchronized by waiting for some, or all, of them to complete before proceeding or terminating the parent script.

     1  #!/bin/bash

     2

     3  sleep 10 &

     4  pId1=$!

     5

     6  sleep 2 &

     7  pId2=$!

     8

     9  sleep 3 &

    10  pId3=$!

    11

    12  echo "waiting for jobs to complete"

    13  #wait ; #--wait for all jobs to complete

    14  wait $pId1 $pId2 $pId3 ; #--wait for specific jobs to complete

 

In the above script, a sequence of sleeping {10, 2, and 3} seconds are executed in sequence, each in the background.  The longest running command  (e.g. 10 seconds in this example) should take 10 seconds, the other two commands concluding earlier.

We can test the proper exection, the script completing in ~10 seconds by timing the execution as follows:

$ time ./foo

waiting for jobs to complete

 

real    0m10.008s

user    0m0.003s

sys     0m0.004s

$

bash-4.1$

Tuesday, March 31, 2026

Python Distribution Error (TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero')

 Last year I cut my teeth at creating a Python package; Dividere


Recently, building the package was encountering an error:

 

 

python3 setup.py sdist bdist_wheel
running sdist
running egg_info
creating dividere.egg-info
writing dividere.egg-info/PKG-INFO
writing dependency_links to dividere.egg-info/dependency_links.txt
writing requirements to dividere.egg-info/requires.txt
writing top-level names to dividere.egg-info/top_level.txt
writing manifest file 'dividere.egg-info/SOURCES.txt'
reading manifest file 'dividere.egg-info/SOURCES.txt'
adding license file 'LICENSE'
Traceback (most recent call last):
  File "/home/lipeltgm/dividere/setup.py", line 15, in 
    setup(
  File "/usr/local/lib/python3.10/dist-packages/setuptools/__init__.py", line 117, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 186, in setup
    return run_commands(dist)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/core.py", line 202, in run_commands
    dist.run_commands()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 983, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 999, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 1002, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/sdist.py", line 59, in run
    self.run_command('egg_info')
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/cmd.py", line 339, in run_command
    self.distribution.run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/dist.py", line 999, in run_command
    super().run_command(command)
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/dist.py", line 1002, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/egg_info.py", line 312, in run
    self.find_sources()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/egg_info.py", line 320, in find_sources
    mm.run()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/egg_info.py", line 548, in run
    self.prune_file_list()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/command/sdist.py", line 162, in prune_file_list
    super().prune_file_list()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_distutils/command/sdist.py", line 380, in prune_file_list
    base_dir = self.distribution.get_fullname()
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_core_metadata.py", line 272, in get_fullname
    return _distribution_fullname(self.get_name(), self.get_version())
  File "/usr/local/lib/python3.10/dist-packages/setuptools/_core_metadata.py", line 290, in _distribution_fullname
    canonicalize_version(version, strip_trailing_zero=False),
TypeError: canonicalize_version() got an unexpected keyword argument 'strip_trailing_zero'
make: *** [Makefile:26: buildPipPackage] Error 1

Apparently, there was an error introduced in a recent version of setuptools.  A work-around is to downgrade to an older version to temporarily resolve the issue.

 

~/dividere$ sudo pip3 install setuptools==70.0
 

 

 

 

 

Tuesday, March 24, 2026

Extending Video Introduction

 Sometimes it's useful to slow, or extend the introduction of a video.  For example, suppose you're utilizing a slow video transition (like a fade out/in effect) but don't want to miss the beginning frames of the second video.  By extracting the first frame of the video, elongating it to X seconds we can preserve the transition effect without losing video content.


Let's take a peek at how to accomplish this;


$ cat -n Makefile
     1    all: postVideo.mp4
     2   
     3    video.mp4: BigBuckBunny.mp4
     4        ${SH} ffmpeg -i $< -codec copy -strict -2 -t 10 $@
     5   
     6    image.jpg: video.mp4
     7        ${SH} ffmpeg -i $< -vf "select=eq(n\,0)" -q:v 3 $@
     8        ${SH} display $@
     9   
    10    preVid.mp4: image.jpg
    11        ${SH} ffmpeg -loop 1 -i $< -f lavfi -i aevalsrc=0 -t 3 $@
    12   
    13    postVideo.mp4: preVid.mp4 video.mp4
    14    #    ${SH} ffmpeg -i preVid.mp4 -i video.mp4 -filter_complex "[0:v] [0:a] [1:v] [1:a] concat=n=2:v=1:a=1 [vv] [aa]" -map "[vv]" -map "[aa]" $@
    15        ${SH} ffmpeg -i preVid.mp4 -i video.mp4 -filter_complex "[0:v] [0:a] [1:v] [1:a] concat=n=2:v=1:a=1 [vo] [ao]" -map "[vo]" -map "[ao]" $@
    16   
    17    clean:
    18        ${RM} *.jpg
    19        ${SH} find . -name "*.mp4" -not -name "BigBuckBunny.mp4" -delete

The input video (video.mp4) is generated by grabbing the first 10 seconds of BigBuckBunny.mp4; refer lines 3-4

Then, the first frame of the input video (video.mp4) is extracted and saved as image.jpg (ref lines 6-8)

Then, a new introduction video clip is generated by converting the image into a video clip of X seconds; ref lines 10-11).  Note, a null audio track is created to preserve the audio in the outgoing video.

Lastly, a new video is created by concatenating the intro clip with the original clip, the result as a video with extended first frame. 

One other trick worth mentioning with this makefile, I frequently want a clean target to clobber any video files created along the way.  However, you don't want to delete the original source file, this can be accomplished by utilizing a find+not condition; ref line 19.  This proves useful for many projects.

And, just like that, you've got a video with an elongated intro.

Tuesday, March 17, 2026

Youtube-Dl Broke/Fixing

 'Some folks' use youtube-dl on a regular basis, to pull raw video and create new content.  Early in 2023 it appeared to stop working.  


$ youtube-dl https://www.youtube.com/watch?v=8QWjCzULyNA
[youtube] 8QWjCzULyNA: Downloading webpage
ERROR: Unable to extract uploader id; please report this issue on https://yt-dl.org/bug . Make sure you are using the latest version; see  https://yt-dl.org/update  on how to update. Be sure to call youtube-dl with the --verbose flag and include its complete output.


Forums seemed to imply the issue was known and a fix was in play, so I put my feet up and figured I'd check back in a few weeks.  A month, or two, goes by and reinstallation/retrying didn't seem to resolve.  So I revisited the fix posts and found it referenced a different package/utility so I tried that out.


$ sudo pip3 uninstall youtube-dl

$ sudo pip3 install yt-dlp


$ yt-dlp https://www.youtube.com/watch?v=8QWjCzULyNA

Hope this helps someone else.

Cheers