Published My First Python Package

 

 

I started dabbling with Python back in 2012'ish, using it pretty regularly over the years but generally keeping my projects close to home.  Recently, I dipped my toe into publishing a Python package, out to the known universe.

Back in the late 90's, the Precambrian Digital Age, I took a couple courses that continue to pique my interest time and time again.  Parallel processing was primarily constrained to supercomputers like the Cray-1 that was homed in a nearby lab on campus, on full display behind a full-glass wall.  A workhorse which eagerly awaited computationally intensive parallelized programs. 

A customized version of Fortran, its vocabulary, the Computer Science department rarely used it, aerospace and atmospheric sciences most heavily used the system.

The second course, Distributed Operating Systems, taken a bit later seemed to pair well with this budding interest in high performance computing.  Beowulf clusters, commodity-grade networked computers running Linux, could be created from RadioShack-provided equipment fueled by inspiration.   Cloud computing, virtual machines and even network-intensive applications hadn't breached the digital horizon, but small-cluster networked labs provided inspiration that one day multitudes of computing assets would one day join hands in forming highly networked, parallel, distributed systems that can be considered common today.

While robust and reliable distributed systems are highly sought after, engineering them is plagued with challenges.  Failed requests could be due to loss of the sent message, the loss of the response, the destination service abruptly terminating, relocation of the service, a over-tasked memory/cpu that slows the response,....or any number of other factors.  Python and ZeroMQ pair well to allow the creation of a distributed system framework which inspired my budding project.

dividere UG

The public project repository is located at:

https://github.com/lipeltgm/dividere

This is my first cut at publishing a python package, I tried to apply good design, test and documentation principles along the way.  One particular challenge I encountered is that the package dependencies require a version of Protobuf that isn't currently available via 'normal channels'.  I'm hoping in time that complication will self-correct when compliant versions become the default.

Until then, it likely will require manual installation of protobuff-v3.19 (or later) before installing via pip3 from pypi:

https://pypi.org/project/dividere/ 

$ pip3 install dividere

With the foundation in place, I'm intending on extending the framework to support more reliable messaging, database components, robust failover detection and recovery.  

More to come in the future, fingers-crossed.