DEV Community: TaiKedz

I am so done here. The slop is overwhelming.

TaiKedz — Mon, 02 Feb 2026 12:15:46 +0000

I started on dev.to because it seemed like a good place to engage with a tech community that didn't feel gamified and ad-ridden.

Then so many flood accounts posting patently incorrect ill-applied CompSci 101 concepts started appearing. Then the scam accounts. And the short posts designed to funnel to Medium and other sites for monetization.

And of course now nearly every sponsored event is some AI slopfest challenge.

I started computing because I like building stuff and solving problems. Things that would remain, skills of my own I could take forward.

AI has this tendency, both in programming and in art/writing to promise this: you can get the aesthetic for free, so long as you don't care about the craft. You don't need to get it, just continue paying a subscription, don't sweat the rest.

It's the microwave freezer meals of our domain; where some of us came to cook, the industry just wants us to consume and dispose of. People using it at home is bad enough, and that'stheir choice; but then restaurants start doing it - and suddenly even the professionals aren't even cooking.

I had a chat recently with an AI programming enthusiast, their take was that software made predominantly with AI didn't so much need maintaining, just regenerate new software. Imagine that! Disposable software! Draining how much freshwater, burning how much fuel, every time you need tk regenerate a website for a customer?

What the actual duck.

I am done here, Dev.to . It was a nice idea. I'm sure the community will grow, and I'll be a dinosaur fossil, and all that.

I'm just so sorry it happened.

Shell/Python vs Go - practical thoughts form a reimplementation project

TaiKedz — Mon, 09 Jun 2025 13:39:50 +0000

This is perhaps not your typical "Golang vs (language)" post... I've seen it said in various places (citations lost, please roll with it) that Go is a good replacement for shell scripting. At first, I was highly skeptical - Go is a compliled, stricly typed language with more boilerplate to "Hello world" than needed in any scirpting language. But as a compiled language, it does make sense for portability, and as a properly and strictly typed language, it makes performing complex operations easier to implement... so it was worth trying out.

TL/DR:

Go is a credible alternative to python and shell, for me
I did Alpacka/NG as an experiment to convert form shell to Go
Publishing Go modules for sharing code is super easy
Establishing supply chain trust is an issue
Shell and Python still definitely have their places

Alpacka, as a bash script

Several years ago when I was still a bright-eyed and bushy-tailed distro-hopper, I kept on running into the same problem: "which package manager command and keyword combination do I need on this distro again??" I wrote a simple script I called Alpacka and whose command I rendered as paf (naming choice lost to history at this point) to help me. It supported a variety of package managers, and is still available on gitlab as Alpacka

It had however three notable drawbacks:

It depended on the local commands, which varied in flavour and version over distro and distro versions, of which bash
Bash's syntax is not oriented to complex data structure manipulation
Predictable modularisation of source code is hard without workarounds

The version issue was multi-fold: not only was it a sequential version problem (1.0, 1.1, 2.0, etc) but also an implementation problem (BSD variant, GNU variant, Busybox variant) for which outputs and flags would often vary in key places. Depending on given behaviours and outputs of these would be tricky sometimes, and limit the available operations to the common base denominator. At the time for example, I think Busybox did not fully implement Perl Compatible Regular Expressions (PCRE) in its grep implementation (or was it sed?) so implementing certain actions was difficult, if not downright impossible.

Shell scripting limitations

The lack of rich types like maps, and the inability to return arrays (need to pass around in global variables, yuck, pollutes variables space), prevented things like function recursion (possible, with hoops, caveats, boilerplate and metaprogramming) and key-value lookup (perhaps possible, but too tedious to warrant implementing). For example:

greetings() {
    for name in "$@"; do
        echo "Hail $name!" # no return - just echo to stdout
    done
}

# use `()` notation to force an array, split on whitespace
phrases=($(greetings Alex Jay Sam))

# Access an array using `"${varname[@]}"`
# quotes and extra chars mandatory - else various alternative effects may happen
for phrase in "${phrases[@]}"; do
  echo "We say, $phrase"
done

# Problem - prints:
# We say, Hail
# We say, Alex!
# We say, Hail
# We say, Jay!

To get away from the effect of the automatic splitting on whitespace, we must use a global variable array

greetings() {
  # also, make `name` non-global (all vars are global by default)
  local name

  # Set a global variable for the caller to access afterwards
  PHRASES=(:) # old bash compatibility
  # arrays cannot be empty, so this array is initialized with a single member, the string ":"

  for name in "$@"; do
    PHRASES=("${PHRASES[@]}" "Hail, $name!")
  done
}

greetings Alex Jay Sam

# Use array items from 1, upwards: `${varname[@]:1}`
# to blat the initial placeholder
for phrase in "${PHRASES[@]:1}"; do
    echo "We say, $phrase"
done

# prints
# We say, Hail, Alex!
# We say, Hail, Jay!
# We say, Hail, Sam!

Does this make shell scripting "inferior"? No! The whole point of POSIX and POSIX-like shells is to be command-driven, and scripting enables re-running commands with some extra flair. I've talked about this before. Look at this:

# a contrived example, point is about the processing of program outputs, command-by-command
docker images | grep '<none>' | sed -r 's/  +/ /g' | cut -d' ' -f3 | xargs docker rmi

which

runs docker images
looks for <none>-labelled images
extracts the third field (the ID) line by line
and converts the above into arguments for docker rmi

When you're familiar with shell syntax, the above is easy to write - I did it in under 20sec. Doing the same with a more generalist programming language would take at least multiple lines, and if you wanted to keep it maintainable, a couple functions too.

But for rich data processing, shell scripting is not typically the best choice.

As for other gripes ...

For the modularisation issue, in the script-based version of Alpacka I leveraged another tool I had, the "bash builder" suite, which transpiles a custom syntax and resolves files from a BBPATH library path, into a single deployable bash script. Again, I had to code this solution as a workaround to limitations with shell scripts. Again, shell scripts have idiosyncracies for gearing towards shell usage, not large-codebase usage. It has been fun to implement, but limitations still remain.

Shell scripting argument parsing is pretty convoluted - you can use the opts utility which behaves in mind-contorting ways, or reference positionals - a function's arguments are accessed the same way the script's arguments are, via $1, $2, etc which prevents a function from directly accessing script args. It's possible - but there's a lot of faff to get it to work.

Thus, the Alpacka/NG Go rewrite was born.

Why not python?

I've written a fair bit of tooling python. It would have been my go-to for this kind of project (after my "bash as main language" phase...), but not for this.

One of the issues with shell scripting remains with python: versioning, and command availability. At least it's not a question of variant usually, but between a newer script on older python (feature not available) or an older script on newer python (feature deprecated and removed), there's plenty of external dependency-ing that can go wrong.

At work we got around this by using pyenv , which went some way to working around the issue, but in some instances, pyenv would not even install (distro too old!)

Compiling to binary solves this issue by just not having runtime dependencies (or so I thought - more on that later....!). But it holds by and large.

This is more of an issue when trying to ship code to machines not under you control - in an enterprise environment, mandating specific distro versions and python versions etc is easier. Developing for the wilderness is another matter, and is the use case I have in mind for Alpacka - so it tracks. And if I need to install things first to support Alpacka correctly, purpose is defeated!

Enter Go

Over the last few months I have been toying around with Go and only in the last month been actively trying to use it fully. Re-writing Alpacka was a good fit for me because

it compiles to single deployable file. I want to be able to just download it and start using it, no environment set up
I want a set of package lists (package spec file) that apply under conditions, and I want alpacka to take care of it

The single-binary from Go answers the first, and the fact that I could not have the packages spec was due to the lack of complex types in Bash.

Package spec

The spec I wanted to implement for looks something like this:

alpacka:
  variants:
  - release: VERSION_ID<=22, ID=~fedora
    groups: common, fedora
  - release: ID_LIKE=~debian
    groups: common, debian
  - release: ID_LIKE=~suse
    groups: common, debian

  package-groups:
    common:
    - mariadb
    debian:
    - apache2
    fedora:
    - httpd

Note the catering to different names on different distros, and common names to share on all distros. Handy when hopping and wanting everything covered, right?

How it went

Sub processes

First off, running a sub-command in Go is pretty easy - a little more involved than in shell, no better or worse than in python. Because I did it a lot in this script (calls to package managers), I wrapped it in its own struct with some niceties: runner.go :

// .OrFail() comes from the Result object, whih allows a shorthand exit-now to cut down on verbosity
// -- because these calls would happen A LOT.
RunCmd("apt-get", "install", "-y", "htop").OrFail("Could not run installation!")

I didn't end up needing to pipe anything - but rather than "grep" and "cut" and "sed", instead we have actual programming functions that can operate on data types. It is indeed possible to use a subshell and write a command pipe, but I did not have need for it.

Argument parsing

Next up was parsing arguments. I've written before about some of my basic frustrations with out-of-the-box Go, one of which is around argument parsing. I've since solved this issue for myself in the form of my GoArgs package which offers short-flags and long-flags, positional argument unpacking, and additional handy options such as mode-switched flags.

I know it's a solved problem, but I wrote my own argument parser myself because

supply chain security considerations - if I can write it myself, why open myself up to supply chain attacks
go language self-learning - re-writing Alpacka was not my first Go-venture ;-)
go version releasing - I wanted to learn how to release a go module for re-use (and I did!)
customisation - I had a specific idea as to what features I wanted, and what API I wanted, and one is best served by oneself. I did have to add customisations as I went about writing Alpacka-ng

Publishing a Go module via hosted git repo

Publishing a Go module is as simple as using version control (typically Git, not sure what else is supported) and tagging a commit with v<x>.<y>.<z> - explicitly it needs the v in the front, and three numbers. Go does the rest, and retains a hash for repeat-deployment verification. Note that the library project needs to declare itself with its intended publishing URL: set it up with go mod init github.com/user/project

A project depending on that module then just needs to add as it usually would - e.g.

go get github.com/user/project@v1.2.3

... and then start using it

import (
    "github.com/user/project"
)

func main() {
    project.CallFunction()
}

For comparison, Python has pip/conda/uv for adding external modules. Publishing them uses a setup.py which is unwieldy, but so long as you have a template for re-use, it's easy enough. You can add similarly https://github.com/user/project@tag to a requirements.txt file to directly depend via git repo.

Yaml and Supply Chain Trust

Finally, I could implement the package list spec which I designed around YAML. First issue was to find a YAML package because Go does not ship one by default in the standard library. I've found that many languages do this - support JSON, but not YAML. Why? I know JSON is a very useful representation to serialise data, but YAML is a much more comfortable format for similarly serialised data that also needs to be human editable.

Plenty of tutorials exist online, all pointing to gopkg.in/yaml.v3 without a hint as to explaining

what is gopkg.in? why should I trust it?
who wrote yaml/v3? why should I trust them?

After some digging I found that the official Go project itself recognises gopkg.in even so far as to making certain accommodations for its version notation. This is (at least in part) due to the fact that the site was the main solution for consistent package naming and version notating before the Go project implemented modules. It is a redirector that points to Github repos with a distinct short URL. Repos with a gopkg.in/<PKG>.<VER> can be seen as an "official"/"blessed" package, whereas gopkg.in/<USER>/<PKG>.<VER> are "unofficial"/"we don't verify these guys" sources. gopkg.in/yaml.v3 is an "official" package then, although the Github project it points to has been archived (read-only) since April 2025 (this year). Who knows what shall happen next...

The official contents are published by Canonical Inc, the company behind Ubuntu. Based on this sleuthing, I was finally able to satisfy that the package was not completely random, and my "trust" in the "supply chain" is... acceptable. I now have a target, and a hash, but I'm pretty sure that can be circumvented by a later update on the v3 branch if the redirect gets modified... ah life on the open Internet...

Supply chain attacks are on the rise, the NPM leftpad incident not only highlighted the yanking of packages, but the ability of anyone to subsequently takeover the name of de-published packages. Similarly, typo-squatting can lead to compromises lurking in the deep or just outright hosing a system.

Beyond that, Unmarshalling Yaml/Json is a tad more cumbersome than parsing it in Python and descending dictionaries, but it does have the advantage of ensuring types all the way down.

And then... I was done.

Checking it on Alpine

One of the compile targets was Alpine, though I ended up dropping this target since a/ Alpine's package manager is easy and b/ it is usually only used in Dockerfile specs/IaC in the first place.

I discovered in this exercise however that binaries compiled on a GNU/Linux will not run on Alpine, due to linkages against different system runtime modules (different libc implementations I think?). Huh. A runtime dependency in a "statically" compiled binary. There are nuances to everything after all.

The result then being that an Alpine version of a Go program must be compiled in an Alpine environment, otherwise you get a cryptic bin/paf: not found error, despite the file being indeed there...! Using containers for build helped with that.

Conclusion

Looking back at my work, it seems I have been able to implement this over the course of 8 days, a couple hours per day (and one full day in the mix too). I became more at-ease and fluent with the language as time progressed as one would expect as well. Could I have done this faster in python? Probably, but I would still have a runtime dependency. Could I have done it in bash scripting? With the added manifest, not a chance.

It feels to me that Go is a credible alternative to both shell scripting and python given the requirements. However, I would still say that, with fluency, one would go faster with shell scripts for simple items (the build and install scripts are still shell), and faster in python for "throw it together" operations.

Only You, however, can determine what's throwaway. At the risk of coining an abomination:

💩 Since bad code obviously needs rewritten, write bad python to get going; and if it must indeed grow: scrap it, and write good Go in its place. /jk

😱

Happy coding!

Humanness at Work

TaiKedz — Wed, 21 May 2025 15:49:19 +0000

I want to take a moment to talk about One-to-Ones at work. Send this to your manager. Send this to your director. It's important.

When I took a role several years ago, I was introduced to a concept that blew my mind and continues to reverberate to this day: my manager scheduled a one-to-one with me every week and kept to it - and if meetings caused a reschedule, he would set it for as soon as possible.

Sometimes we would talk about my progress, or how I talk to people.
Sometimes we would talk about my expectations or frustrations.
Sometimes we would talk about company directions or areas for my progress.
Sometimes I would talk about what I needed from the organisation.
Sometimes he would check in with my mental health and remind me that I could also be mindful of self care.
And sometimes. He would keel me over with a litany of dad-jokes, we'd bond over the coffee machine about his llamas at home and his latest anti-fox solutions for his chicken coop.

It was regular. It made us more than colleagues. It made us human to eachother.

Especially in a world where remote/hybrid work is becoming the norm, having regular check-ins, be they professional or casual, are of vital importance.

A weekly half-hour one-to-one with the boss keeps things human. A boss who knows how to be human is a keeper; one that just uses the exercise as a tick-box and ammunition is to be dropped. In my jobs, I have made a point of gathering team-mates over lunch - even to the point of everyone buying their sandwiches, and coming to eat around a desk. Discovering each other as humans goes much further than any ill-designed short-noticed "staff night out" or "team building event."

In general for colleagues, a more personable one-to-one approach is advisable: every week, have a half-hour date with one of your colleagues. Be it going out to buy the sandwiches together, or indeed eating the lunch together, or just simply a half-hour call if remote teaming. Make a point of it. This week Alex. Next week Sam. Go beyond the transactional Teams call.

Households, teams, organisations, operate better when everyone sees each other humanly, genuinely. Nobody needs to agree on all points - but being able to understand each other at human level allows us to work better together. And who knows - maybe find new friends.

So set up a one-to-one with a colleague now. And for everyone on your team. You don't need HR to "Approve" this... or if you do... well... let's just say there are better companies out there.

What ((programming) language) should I learn this year, 2025 ?

TaiKedz — Wed, 15 Jan 2025 17:01:46 +0000

It's a question that is evergreen, but especially prevalent at the turn of the year: what skills should I learn this year?

It's not just programming languages either, there are a slew of areas of life where skills are desirable but you wonder how to go about it or whether it's worth the effort.

And if there's anything I've learned it's not "should I learn it" but "how much does it cost to learn it?"

TL;DR - just set aside time to learn any that you feel curious about. It costs you little, and you gain so much from learning to think in different paradigms.

💸 Cost of starting

(I know, I am digressing, bear with me ...)

Some skills are more expensive to pick up, and wondering whether you want to spend a lot without even knowing if you'll be sticking with it is a reasonable reason for pause.

For example, if you wanted to pick up a musical instrument skill, there's the outlay of buying or renting said instrument upfront. To learn piano, you need regular access to a piano for practice, and those things don't come cheap, and going all the way to a practice hall every time can quickly curb a beginner's enthusiasm; so it's fairly reasonable to um and ah about it. But if you wanted to pick up an instrument with a piano-layout keyboard (albeit limited), you could probably manage a melodica and see if the layout works for you - it's cheap, it's portable, and a lot of beginner piano theory and tutorials apply to it.

Conversely, there are skills which cost little to get started with - cooking certain types of foods if you already have a stove/oven, or take up drawing (pencils and eraser, and a stack of A4/letter paper is easy to come by), and then you can just scour youtube for a variety of tutorials - try drawing chibis to start with?.

Similarly there are many options and approaches for learning foreign spoken languages, ranging from free (apps and meetups) to pay-for (courses) or even specialization (degrees) which you can approach little by little as you find your feet.

Last year I took up foraging and fermenting, and whilst there was a little cost in books (I have many now) and jars (I just bought some more the other day), it was mostly a cheap outlay for a set of skills I take great joy from today. Mostly, gaining a reason and actual pleasure from heading out into the woods, even on a rainy day.

It's not just about computers - anything you can do to spend time away from the screen is immensely important for your continued good health to come back refreshed 🌲

💻 What Programming Language?

But back to dev talk. What programming language is worth learning ?

I'll cut to the chase and answer this: ⭐ any you feel curious about ⭐ . If you can set aside 30min in any given day, or every other day, you'll get enough of a flavour of a language to decide whether you want to pursue it. Heck, dedicate an hour to the language and its base systems, and if you're already mad at it after that, just stop, and uninstall.

Because, with very few exceptions, getting started with a new programming language is near zero cost, in monetary expenses at least. Aside from the time you need to set aside, if you already program, you have the required equipment and skill to just launch into it.

Sure, it may pay to read up on the languages that have piqued your interest upfront, and once you've done that you will probably have a good idea which ones you do want to dedicate an hour to and which are already just outright not a fit for you.

And if you come away with a shortlist thinking "this one or that one or the other," I would wager that the obvious answer should be: all of them , in sequence.

✂️ Narrowing Down

It also helps to know what kind of ballpark use case you are aiming for. My use case requirements are

command line
ideally, compiled
preferably without external runtime requirements

My other use case (as a separate effort) is

back end for simple websites/apps
simple/minimal build and hosting requirements
good standard library/reduce external dependencies

Last year I doubled-down on Zig, as idiosyncratic as it is to me coming from scripting languages (and very much because of it), and I am pursuing Go as well, as a nicer middle ground.

Between what I have experienced from C and Zig, I am not even considering C at this point anymore. Maybe later, but I do not see the benefit, given my language selection as it stands. I had a quick look at Kotlin from afar (perusing how-tos and videos, and simply reading a quick install guide) and decided I had no interest.

For web, PHP was also on the docket for a short moment, but I don't feel I want to go there yet; I feel more relevance looking first into Django and Flask before I return to PHP (I used to do php3 projects, way back when...). Gleam and Scala run on dedicated VMs and whilst in a production setting that's not a problem, I do want to make applications that can run on free-tier hosting, which PHP has had the advantage since the 90's ...

🔨 Mini-projects

Once you get past the first hour of each language, I reckon there will be some you'll think "nope, I'm done here" - and that's fine. That's the whole point of this in the end. After which you're left perhaps with one or two languages that remain contenders for Your Next Thing. What next?

One of the questions I've heard from a couple of friends repeatedly is, "how do you even choose a side project?" My take is, what's a thing you wish there was a program for? What's the simplest thing that program would do?

I live on the command line, so this question is usually easier for me, barring having to do more complex things (I don't (yet) do media processing, or have need to do rich GUI stuff). But I do often want to say organise and deduplicate a few reams of photos or MP3's (my backup drives are in disarray since forever and I need to wrangle them to sensibility...) , and I am often wishing websites would publish and list their content sensibly (web scraping and RSS feed corralling comes to mind).

Try to think, how would a command line program do 80% of what you need to do? Or for GUI projects, how would you get the thing done with a couple simple widgets and buttons? And take it from there.

Aim to scratch a particular itch 😖, reduce the feature set to minimum 🔪, and have at 💪.

(As an example dump: I am currently trying to rewrite my bash-builder project (extending the lenker), I have a tarball-dependencies project inspired by Zig's build.zig.zon file, did an /etc/os-release reader for something simple, and pondered a PATH management utility . For web stuff, I have a player registry I ideated once when thinking about cross-server stuff for Luanti, but I have yet to properly sit down to. All of which I will likely try revisiting in various languages as the need arises.)

📋 What languages for me this year?

If you really want to see a language list from some nobody on t'interweb, here's where I'm at - this year I intend to:

Continue with Zig and Go - I want to gain familiarity with a systems programming language, and a systemsy language that still has some extra fluff.
Reignite my Lua skills with Love2D, and maybe return a little bit to Luanti
Get more hours into Django (yeah not a language, but furthering my Python mastery)

🎉 Voila

It's a relatively low cost to entry to try out a programming language when you already have a computer and an Internet connection, and well worth sinking an hour or two each into multiple languages and trying to understand why they work the way they do, and what new ways of thinking they can bring to you.

It's not "should I try X or Y" but "I've tried X and I've tried Y - how much more time shall I spend on each?" ⏲️

Simple Python Logging - and a digression on dependencies, trust, and Copy/pasting code

TaiKedz — Tue, 26 Nov 2024 12:18:36 +0000

Header Image (C) Tai Kedzierski

Goto Snippet

This post is opinionated.

Python's default log setup is unhelpful; it works against the "batteries included" approach we have come to expect.

From a useful log message, I want to know when, what level, and what information. I may want it on console, I may want it in a file.

This should be simple - but in Python I end up every time having to look up how to create a full logging utility with custom file handling and string formatting.

It should be as simple as logger = getLogger(), but the default behaviour for some unknown reason is to provide a completely useless formatting, and no shorthand for a sensible default.

That or I need to download some pip package of unknown provenance, trust that it hasn't been name-hijacked, or doing some obfuscated exfilration. The leftpad incident from 2016 comes to mind, as well as the Revival hijack attack from 2024 which was essentially the same problem in a different repo system.

In fact, any user-repo without namespacing is vulnerable to this: Node's npm, Python's pip, Arch's AUR, Canonical's snap ... to name a handful who just let users upload whatever. Even namespacing isn't a guarantee of trust - I've come across projects that distribute their software through these channels not through the project's name, but via some arbitrary dev's monicker, raising doubt as to the authenticity of the package. I gave my thought process on how to decide on whether to trust a source in a previous post on using syncthing in a work environment.

External dependencies in user-controlled repos are the devil, and should only be considered when the solution to a problem is complex. And in general, simple solutions should just exist directly in the code base - ideally self-written, but sometimes the problem just strafes into the "cumbersome enough" space to make a dependency feel both reasonable and icky.

The answer: write it once, stash it away in a Github gist or in a "useful snippets" repo of your own. Copy and paste.

Copy Paste? Ew!

"Copy and paste" of code probably sends alarm bells ringing for any seasoned coder. "Don't repeat yourself," "use a package manager," "write once, update everywhere." These are good instincts to have, but case-by-case, it is also good to know when copy-paste is preferable.

In this case, the requirement is to avoid unnecessary external dependencies for a simple solution to a simple need . In leftpad as with this mini-logger, the required code snippet is short and easy to understand ; it is no loss to reimplement if needed. It is also appropriately licensed (yes, it may be just a snippet; it remains however recommendable to ensure that what you are copying is indeed allowable. Be wary of copying random blobs of code.)

Mini Logger Snippet

I include below a code snippet for a mini-logger utility which allows for a single call with minimal configuration:

from minilog import SimpleLogger

LOG = SimpleLogger(name="mylog", level=SimpleLogger.INFO)

LOG.info("this is useful")

Which prints to console:

2024-11-20 10:43:44,567 | INFO | mylog : this is useful

The mini-logger code

Copy this into a minilogger.py file in your project. Tada - no external dependency needed. Left untouched, it will remain the same forever. No name hijacking. No supply-chain injection.

# For completeness:
# (C) Tai Kedzierski - Provided under MIT license. Go wild.

import logging

class SimpleLogger(logging.Logger):
    FORMAT_STRING = '%(asctime)s | %(levelname)s | %(name)s : %(message)s'
    ERROR = logging.ERROR
    WARN = logging.WARN
    INFO = logging.INFO
    DEBUG = logging.DEBUG

    def __init__(self, name="main", fmt_string=FORMAT_STRING, level=logging.WARNING, console=True, files=None):
        # log level here sets lower limit on all logs - hnce hard coded to DEBUG
        logging.Logger.__init__(self, name, logging.DEBUG)
        formatter_obj = logging.Formatter(fmt_string)

        if files is None:
            files = []
        elif isinstance(files, str):
            files = [files]

        def _add_stream(handler:logging.Handler, **kwargs):
            handler = handler(**kwargs)
            handler.setLevel(level)
            handler.setFormatter(formatter_obj)
            self.addHandler(handler)

        if console is True:
            _add_stream(logging.StreamHandler, stream=sys.stdout)

        for filepath in files:
            _add_stream(logging.FileHandler, filename=filepath)

The MIT license essentially allows you to "do whatever you want with this." No strings attached.

There we are. A simple log 🪵

Learning Zig : a Python dev's learning notes

TaiKedz — Sun, 13 Oct 2024 22:15:48 +0000

Cover photo (C) Tai Kedzierski

Scripting languages and System languages are two parts of a vast ecosystem. I wanted to cross to the other side to see what life is like there...

Having trawled through a lot of reference material, I found some common-use concepts still lacking for the beginner, so this aims to stand as a jump pad for anyone else from a python/scripting starting out with systems languages in general, and Zig in particular.

Zig? ⚡

Zig is the ace new kid on the block and, whilst still in its early days, is proving that it has a lot to offer. The release of the bun JS/TS runtime launched its name to the fore and developer interest has increased rapidly over the last two years - it debuted on StackOverflow's 2023 survey at an impressive initial 39.5% on the "admired" rating metric, and the following year rose to 73.8% - well ahead of C's 47.4% , and only just tailing Rust's 82.2%.

I myself work predominantly with Python, a garbage-collected scripting language with lots of QOL features, and have been dabbling with Go (both sitting at a tidy 67% rating for the "admired" metric). Both occupy a same space of wanting to facilitate rapid development - but whereas Python offers a rich language feature set for flexibility and readability; Go offers a much more stripped back syntax with a focus on simplicity and efficiency.

I've coded in Go a little bit, and was considering basing my next efforts there, as it feels much more approachable for a script language specialist; but I felt it would be a much better horizon-widener to dive properly into a systems programming language. I tried to get on board with Rust, but felt its hype and promises ended up not delivering enough against the difficulty to work with it. The worst for me was the amount of code nesting its management entailed, especially managing error cases against option-matching.

Documentation and Learning Resources

There are a number of good learning resources for the basics - the order in which I stepped through was to first read the language reference. Following some advice I saw for Rust, I read through the documentation first whilst resisting the temptation to experiment too much.

Language reference

I then started looking through the Ziglings excercises one by one, and finding them to be pretty good at gently easing the learner through basic concepts. After exercise 25 or so, I decided that perhaps I didn't need to go through the rest of it and decided to rewrite my Go project Os-Release Query . I had written it in Go as a beginner in about two or three sessions to completion, so I reckoned it was a simple enough to proceed with. It took me about two weeks with several sessions of learning and stumbling.

Ziglings

As I contended with a few teething difficulties, I found myself frequently going to Zig.guide for simple examples and explanations of usage - specifically the ArrayList and Allocators pages, the former being key for doing general file reading tasks. My first mistake was to think about string arrays similarly as in Python or in Go (see "Strings and ArrayLists" below).

Zig.guide

Finally, sensing I was needing a bit more guidance, I found myself stepping through several excellent videos from Dude the Builder's (José Colón) Zig in Depth series. Having read through the standard documentation, and cross-referencing against the standard library documentation, Colón's guidance was the last missing piece for getting proper progress.

And of course, web searching and StackOverflow filled in specific gaps, notably I kept coming back to a string concatenation answer on SO.

Concepts

Arrays, Slices, Strings and ArrayLists

Unless you've worked in C before, you come in for a rude awakening to discover that there is no native language construct for a "string" (beyond a double-quote notation of static literals). Most modern languages have them. All scripting languages have them in some form. Go and Rust have them. But C and Zig share the same principle: there is no string, only sequences of bytes. Whilst C has a "character" char type, Zig takes it a step further by only referencing an unsigned integer type to represent a byte.

# C
char hi[] = "hello"; 
// or even
char* hi = "hello";

# Zig
var hi:[]const u8 = "hello";

On the Zig side, note the const on the array type notation - a lot of functions which receive a "string" as parameter best makes the assumption that the data being received could be a literal - in this case, it's hard-coded and unmutable. If a function is defined as

fn myFunc(data:[]u8) { ... }

then the type cannot be a string literal, but instead requires that the item be mutable, that it can change - pointers into heap memory locations most frequently.

There are many subtleties to this, and I mainly worked against an item from Zig.news to keep myself sane.

I made an early mistake to think about loading a ASCII file lines, into an array or array of bytes, whereas in fact what I needed was an ArrayList - a dynamically growable and iterable sequence of items - of slices, which are sort of 'views' (with a pointer and length) onto arrays (a fixed-length sequence of a given type).

Stack and Heap

Stack vs Heap is a key concept in any language where you are made to manually manage memory, there's no way around it really, and coming in from GC languages, I had to be disabused of some simplistic notions.

Understanding the stack and the heap was a bit more than "heap is big and persistent/stack is small and throwaway". It also goes hand-in-hand with reasoning on passing by value or reference, which in turn is more than "did you pass through a native literal type or a pointer." Geeks for Geeks goes over the principle from a mainly C/C++ point of reference, whereas OpenMyMind.net walks through it specifically with Zig examples.

I am not in a position to explain its nuances myself as I am still learning, but the shift in understanding these was very much crucial for comprehending what I was doing in my project. I can only recommend reading up on those two sites as a starting points as mandatory reading before proceeding further.

Geeks for Geeks - Stack vs Heap with C focus
Open My Mind - Stack vs Heap with Zig focus

Allocators

One of the key features of Zig is its use of allocators, which is one of the major improvements in memory management assistance we get from the compiler and its debug-mode builds. It is still possible to double-free and leak memory from improperly-written code - but using the debug build (the default build mode) causes exceedingly useful stack traces to show the origins of leaked/non-freed memory and give you an early chance of catching and remediating memory problems.

The General Purpose Allocator can be the go-to reference for pretty much anything if you just want to get started. Both Zig Guide and Dude the Builder give an excellent run down on how to use them - once you see them in use a couple of times, they usually feel mostly plug-and-play.

Zig makes a promise - and sets a standard/best-practice - of no hidden allocations: if a function does not take an allocator instance, or a type is not initialized with one, then it promises no allocation to happen. Likewise, library creators are encouraged to abide by the same principle. This goes a long way to ensuring no nasty memory leak surprises deep in a project's dependencies.

Zig Guide on Allocators
Zig in Depth on Memory management

Structs and Methods

Unlike in C, structs can come with their own dedicated functions, or methods, firmly attached to the types they are associated with. Like in Python, if the first argument is a self reference, it can be used on an instance, but without it, it can only be used from an uninstantiated struct type.

This gives the flexibility of some type-specific operations that are carried through a instance.method() notation, but avoids the pitfall of object-orientation inheritance deluges by just not permitting them.

I find this notationally and conceptually convenient, and a good introduction to the idea of "composition over inheritance" that is becoming more and more popular. Stuff that, JavaDeepInheritanceSingletonFactoryInstantiatorHelper!

Build System

The build system in Zig is interesting. I cannot claim to fully understand it much at all, except that to say I am glad I don't have to contend with Makefiles. Whether it is in fact better remains to be seen (for me), but I am going to keep exploring.

One thing I do enjoy very much however is the build.zig.zon file, and how it manages dependencies.

.{
  .dependencies = .{
    .zg = .{
      .url = "https://codeberg.org/dude_the_builder/zg/archive/v0.13.2.tar.gz",
      .hash = "122055beff332830a391e9895c044d33b15ea21063779557024b46169fb1984c6e40",
    },

    .module2 = .{
      .url = "https://site/path/item2.tar.gz",
      .hash = "1210abcdef1234567891",
    },
  },
}

Using other peoples' libs

There is no central package manager in Zig - simply point at tarball URLs online and fetch them. ZigList.org tracks libraries on Github and Codeberg (primary host of Ziglings and a preferred space for some Zig creators), and if you do publish one on either platform, it should eventually get picked up.

The hash however is not a hash of the tarball, but a cumulative hash of each exported item from the package.

Either way, there is an interesting aspect of the system: after performing zig build, any dependencies are downloaded and unpacked to ~/.cache/zig/p/<hash>/... . There are two aspects that I came up against in playing with this:

You cannot easily calculate the hash in advance yourself
If a hash exists already, then the tarball is not re-downloaded.

So when I wanted to add my second module, I needed a placeholder hash. I blindly re-used the hash from the prior module and could not figure out why it wouldn't a/ prompt me for hash re-validation on my bogus hash, and b/ didn't find the files from the second module.

Eventually after trawling the net I found two resources, I figured that I should simply copy an existing hash and then mangle the last few bytes of it. The standard documentation is admitted by the core team to not yet fully be there yet - which I understand since the dust isn't yet completely settled on the final forms of many things, and a lot is still moving about, so other peoples' blogs are a godsend...

Using a library involves adding some code in your build.zig file:

    // Find the dependency named "zg" as declared in build.zig.zon
    const dep_zg = b.dependency("zg", .{});

    // Get the `CaseData` module from the `zg` dependency
    //   and register as importable module "zg_CaseData"
    // In project code, use @import("zg_CaseData")
    exe.root_module.addImport("zg_CaseData", dep_zg.module("CaseData"));

Usually the README of the library will give you more detailed instructions and, hopefully, the hash to use as well. The above was an example for using Dude the Builder's zg library for UTF-8 string handling.

Creating your own lib

One item I created for my project was an ASCII file reader and, knowing that I would likely want to re-use that code, I decided to try to make my own library.

The most minimal snippet for creating a library in Zig 0.13.0 consists of a source directory, a build.zig.zon to declare relevant items, and a build.zig for executing the build when it is imported:

The build.zig.zon file:

.{
    .name = "my-lib",
    .version = "0.1.0",
    .paths = .{
        "build.zig",
        "build.zig.zon",
        "src",
    },
}

The build.zig file:

const std = @import("std");

pub fn build(b: *std.Build) void {
    const target = b.standardTargetOptions(.{});
    const optimize = b.standardOptimizeOption(.{});

    _ = b.addModule("my-lib", .{
        .root_source_file = b.path("src/main-file.zig"),
        .target = target,
        .optimize = optimize,
    });
}

That's it. Follow then the above for importing it.

Final thoughts

It's only the beginning for me with Zig and so far, it has been rewardingly challenging. Going on this little escapade has taught me much about how to design for memory efficiency (I have not achieved it - but I know how my Python-oriented code design would not fly in a performance-oriented space).

And that's growth. 🌱

Beyond "Hello World" - comparing starter code in C, Rust, Python, TypeScript and more

TaiKedz — Thu, 15 Aug 2024 16:33:44 +0000

Cover image (C) Tai Kedzierski

TLDR : The classic "Hello World" doesn't show enough of a language to get a flavour for it - in this post we explore the same example written in several languages for comparison. "Hello Y'All" is a simple example that exercises more of a language. How does the "Hello Y'All" example work in your favorite languages?

Disclaimer: none of the code here is intended to be "idiomatic" or "optimal". I'm a learner especially of the systems languages myself. Any suggestions for correction and improvement are welcome, and possibly even applied if compelling.

The examples in this article have been posted at https://github.com/taikedz/hello-yall . A yet-to-be-done "advanced example" is in the works as well

Hello Y'All basics
Python
Bash
TypeScript via Bun
PHP for Web
C
Go
Zig
Rust
Lua
Java
Groovy
Perl
Extended Bash for Bash Builder
Outro

"Hello World" is the classic example program every new user is exhorted to write: superficially it serves as a way of showing how easy it is to do a basic task in the language; but it also serves as a simple way to check that the base tools are installed.

For example, just making sure python is installed properly, one can write the following:

# some pythonfile like "spam.py"
print("Hello World!")

And just run it:

python spam.py

As a demonstration of how easy it is to get started, it contrasts with the likes of Java quite directly:

// MUST be "HelloWorld.java" file - public class name must match the file name

public class HelloWorld {
    public static void main(String[] args) {
        System.out.println("Hellow World!");
    }
}

Then to compile and run it:

javac HelloWorld.java
# Produces a HelloWorld.class

# To run it though, don't include ".class" !
java HelloWorld

Arguably, it's a good tooling check, but it says little about how easy it is to do some basic things in a language.

The one thing that always costs me time in getting started with any language is navigating its intricacies on splicing functions out to other files locally - and then importing them. Another is just wanting to see a simple example of how some basic flow works in the language: calling a function, returning, concatenating strings, and a simple iteration.

🔝

Hello Y'All

I'd like to propose the following as a good reference example to showcase basic functionality in a single page - one that should be a very basic challenge, but with enough in it to allow a seasoned developer to see the basic features, and appreciate the tutorials properly from there.

So here's a "Hello Y'All" program to implement along the following rules:

The example takes at least one command line argument, and handles the event of no-arguments
The example requires a function to be placed in a second file in a subfolder
The function returns a value
The example loops over CLI arguments, calling one at a time to the function, and each time prints the returned value, concatenated to other text
Include the overall bare-minimum instructions include how to build the program (if necessary) and run it

Why these requirements ? They showcase some of the basic features every language should support, for one; but also if you've already got a couple of languages under your belt, it is helpful to get a more holistic sense of the language's design and approach in one go, in a way that allows consistent comparison across languages.

We will see over the next few examples that splitting code into local files is not always intuitive - every language has its way of handling importing, according to the architectural/environmental constraints of its time and/or target use-case. This is the main reason I was compelled to lay out a base example principle and write this post. Both in Rust and Go, I found myself mired in a hell of answers on "how to import external packages" and needless hunting for the answer to "how to import a local file."

🔝

Python

Python makes importing local files very simple, both syntactically and operationally - part of the "batteries included" approach. For sanity, I recommend PYTHNOPATH point to the directory containing the top level folder, always; it is usually optional as python does try to resolve things automatically, but sometimes it just needs the PYTHONPATH. I only learned recently that __init__.py files were optional ...

Create a ./hey.py file with these contents:

import sys
import stuff.hello as hello

if len(sys.argv) < 2:
    print("No arguments specified!")

for name in sys.argv[1:]:
    phrase = hello.greet(name)
    print( phrase )

Create a ./stuff/hello.py file with these contents:

def greet(name):
    return f"Hello {name}"

To run it:

# Good practice to ensure python knows where to resolve top-level imports from
export PYTHONPATH=.

# Run it
python hey.py Jay

🔝

Bash

Bash is a language, that few really take the time to learn properly. Lots of unwieldy and unmaintainable bash scripts get written, which is a shame since... it typically is also code that holds up your business. Give it more love. ❤️

In ./hey.sh put this:

# Imports the file contents ; all tokens in the file are added to the global scope
source stuff/hello.sh

if [[ -z "$*" ]]; then
  echo "No arguments supplied"
  exit 1
fi

for name in "$@"; do
    phrase="$(hello_greet "$name")"
    echo "$phrase"
done

In ./stuff/hello.sh write:

hello_greet() {
    # Arguments to functions are positional
    echo "Hello $1"
}

Run it:

bash hey.sh Jay

Note that source-ing is specifically relative to the current working directory! Source-ing from an already-sourced file is unreliable at best. For this reason, there is not much of a habit of splitting code into files where bash scripts are involved, and was the impetus behind writing bash-builder

🔝

TypeScript via Bun

I don't think I could get away with not including TypeScript on this list (I am forgoing trying to deal with JavaScript). And I certainly don't want to deal with discussing NodeJS and npm.

Bun makes dealing with the JavaScript ecosystem easier, and removes the need for a lot of the independently existing standard tools. node/nodejs and npm commands can be replaced by bun as a command, and conflicting requirement dependencies don't crop up. As a newcomer to it, do yourself a favour.

Write in ./hey.ts:

import { greet } from './stuff/hello.ts'

if (process.argv.length <= 2 ) {
    console.log("Please supply names !");
    process.exit(1);
}

for(var name of process.argv.slice(2)) {
    var phrase = greet(name);
    console.log(phrase);
}

And in ./stuff/hello.ts:

export function greet(name:string) {
    return `Hello, ${name}!`;
}

Run this through bun:

bun hey.ts Alex Sam Jay Pat

If you need to compile it down to plain JavaScript:

bun build hey.ts > hey.js

Just be warned that in JavaScript and TypeScript, [7,-2,6,-5].sort() == [ -2, -5, 6, 7 ] is a reality. It's not the only madness still around in 2024 in the language. This should have been specified out by now, surely...

🔝

Web Server PHP

PHP has received much hate over the years, but let's not forget that we wouldn't have the Web of today without it (MediaWiki, WordPress and facebook are the notable projects that got a head start with it), and free tier hosting. You can be up and running with a website and even a working proof of concept business using PHP and a free webhosting tier faster than with any other language - instead of chaining yourself to excessively "flexible" billing.

This example takes a slightly different form, assuming the base case is in fact to run as a server script - so we'll retrieve from the GET request's query map, rather than iterating a list.

In ./hey.php write:

<?php
require_once "stuff/hello.php";

if(count($_GET) == 0) {
    echo "Give me a query!";
} else {
  foreach ($_GET as $name => $place) {
    $phrase = hello_greet($place, $name);
    echo "<p>$phrase</p>\n";
  }
}
?>

In ./stuff/hello.php write:

<?php
function hello_greet($origin, $name) {
    return "Hello $name from $origin !";
}
?>

Upload these to your web server hosting root (usually a www/ folder), and visit your site. Assuming it's served at http://hosting.tld/~you then you would go to http://hosting.tld/~you/hey.php?Kim=Korea,Sam=Sweden

This will simply write the greetings in the web page.

🔝

C

Ah C. The gold standard of programming, besieged on all sides by upstarts. Zig, Rust, Go, all vying to be "the next C". And for understandable reason - so many catastrophic bugs and security issues stem from the mismanaged memory and ill-conceived pointer arithmetic.

But, it is still a much needed and much beloved language, and essential in embedded programming. Perhaps Zig or Rust will replace it there, one day. But what is sure is that it is not leaving the party any time soon.

Write a ./hey.c

#include <stdlib.h>
#include <stdio.h>
#include "stuff/hello.h"

int main(int argc, char* argv[]) {

    if(argc < 2) {
        // There is always at least one arg at argv[0], the program's name
        printf("No arguments supplied\n");
        return 1;
    }

    for(int i=1; i < argc; i++) {
      char* phrase = hello_greet(argv[i]);
      printf("%s\n", phrase);

      // For completeness, we note here that any allocated memory
      //     MUST be deallocated - and ONCE only !
      free(phrase);
  }
}

Then write a ./stuff/hello.c

#include <stdlib.h>
#include <string.h>

char* hello_greet(char* name) {
    // Constant, on the stack
    char* base_hello = "Hello ";

    // Because we're returning the data out of a function,
    //   we need it on the heap
    //   The +1 is for the null-terminator on the end of the string
    char* phrase = malloc(strlen(base_hello) + strlen(name) + 1);

    strcpy(phrase, base_hello);
    strcat(phrase, name);

    return phrase;
}

C requires using header files .h - one doesn't import source files directly, but instead you compile all the source files as binary, import the headers as a way of resolving symbols. I don't have the ability to discuss the itnernals of this from a compiler perspecitve, but it is done this way for consistency : if you are including a pre-compiled library from elsewhere, you need a header file to tell you and the compiler what's actually available. I think.

Write stuff/hello.h

// We guard against multi-inclusion by setting a one-time declared name
#ifndef HELLO_H_
#define HELLO_H_

// This tells the compiler that it should expect to find such a
//   function defined in the binary code that is being linked
char* hello_greet(char* name);

#endif // HELLO_H_

Run it:

# See that we _compile_ the source files
#   even though we are '#include'-ing header files
gcc -o hey hey.c stuff/hello.c

./hey Jay

See the pretenders to the throne below ...

🔝

Go

I've seen Go be ascribed a number of lofty goals - a better C (not quite, especially with garbage-collection pauses), a replacement for Python (it's more performant, but much less friendly syntactically), and a replacement for shell scripting (I can kinda see that, if I squint hard enough).

All imports are done from paths along the GOPATH (and from the current project as discussed below) which usually points to a single directory, usually in ~/.go - not entirely sure why that is, as different projects will usually have different dependency needs - but that's the convention. You can of course change the GOPATH as required, but that is neither necessary nor in scope for this example - it should fully irrelevant. If the compiler spits out something about not finding something on the GOPATH, the example (or your copying of it) should be suspected first...

Your project needs to be initialised - in the top level of the project, do go mod init $module_name , for example

go mod init net.myname.greeter

... conventionally using a reverse domain notation.

The simple way of splitting code into several files is to simply write several *.go files, all with package main declarations, and running go build -o my-app src/*.go . All symbols from these files will be directly available in the scope of any main-package file. As a build requirement, all files must be in the same directory. In this mode, it's not possible to store some files into subfolders.

To get past the same-dir restriction and be able to place our greeter in its own folder, we need to create a submodule - specifically we are creating an explicit module under a different package declaration.

In ./src/hey.go write

package main

import (
    "fmt"
    "os"
    "log"
    "net.myname.greeter/src/stuff"
)

func main() {
    if(len(os.Args) < 2) {
        log.Fatal("Please supply arguments!")
    }

    for _, person := range os.Args[1:] {
        var phrase string = stuff.Greet(person)
        fmt.Println(phrase)
    }
}

The module is created in a subfolder, and the main file must share the name of its parent folder.

In src/stuff/stuff.go write

package stuff

// Because this function is to be used from outside the `stuff` package
//    it needs to be Capitalised.
func Greet(name string) string {
    return "Hello " + name
}

Build and run:

go build src/hey.go

./hey Alex Sam

# To just run without storing a build file:

go run src/hey.go Alex Sam

To add more files and functions to stuff/ , simply add the ./stuff/*.go files, all with package stuff declarations. All files with the same package stuff declaration will be able to use eachothers' symbols within the same stuff package.

🔝

Zig

From the little experimentation I've been doing, Zig seems to be the most likely contender in the race to replace C. I've not yet properly sat down to do a mini-project, so this Hello Y'All example is going to be the most complete amount of active work I'll have done...

The whole "allocators" pattern takes getting used to and was arguably the item I had to spend most time on. There are several types of allocators (many more than in that article), and choosing one is no doubt an expression of sagacity. No doubt the neophyte would best be served by using the General Purpose Allocator as they start out. There's a few patterns, most generically the alloc/free pattern. If creating memory for structs, it's var thing = allocator.create(your_struct) ; defer allocator.destroy(thing) . If it's an arena allocator for short-lived one-run programs, arena_alloc.init() the item, and then defer arena_alloc.destroy() the whole thing at the end of the progam.

I do not claim to have grasped the subtleties at this point. The following code may yet be suboptimal. On the plus side... no header files !

Write one file hey.zig:

const std = @import("std");
const hello = @import("stuff/hello.zig");


pub fn main() !void {
    const stdout = std.io.getStdOut().writer();

    // We need to manage the allocator in the very space we are going to call its free/destroy from
    // we can't confine it easily to another function, so it feels a little cluttered ...
    var gpa = std.heap.GeneralPurposeAllocator(.{}){};
    const allo = gpa.allocator();
    defer _ = gpa.deinit();

    const args = try std.process.argsAlloc(allo);
    defer std.process.argsFree(allo, args);

    if(args.len <= 1) {
        try stdout.print("Please specify arguments!\n", .{});
        std.process.exit(1);
    }

    var phrase:[]u8 = undefined;

    for(args[1..]) |name| {
        // Because hello.greet creates the data and chucks it back out, we pass in
        //   the allocator, so that we can retain responsibility for freeing
        phrase = try hello.greet(name, allo);
        defer allo.free(phrase);

        try stdout.print("{s}", .{phrase});
    }
}

And then write ./stuff/hello.zig

const std = @import("std");

pub fn greet(name:[]const u8, allo:std.mem.Allocator) std.fmt.AllocPrintError![]u8 {
    // Pass-down an allocator. All standard library functions which need
    //  heap space will take an allocator, instead of using their own
    const phrase = try std.fmt.allocPrint(
        allo,
        "Hello, {s}!\n",
        .{name}
    );

    return phrase;
}

Then compile and run:

# Compile
zig build-exe hey.zig

# Run

./hey
./hey Sam Pat

🔝

Rust

Rust's documentation is a bit light on the explanations of splitting code out into folders, but I got there eventually - not sure I remember how, so I cannot offer any links.

Rust is very fussy when it comes to types, and probably one of the biggest gripes I have with it is its myriad String/&str/OsString/&osstr and goodness knows what else types. A lot of the complexity in learning Rust comes from thinking about ownership and lifetimes which are made explicit to prevent memory issues.

Largely, it feels like the comment on Lisp: learn it, and it will make you a better programmer, but you probably won't want to keep it as a main language. Not to say that there aren't Rust devs (goodness knows there are quite a few now), but for the common mortal who is not systems-programming, it is more than what is needed to get a job done.

I have included quite a bit of commentary because I feel the need to explain why the example isn't simpler than it is...

For the top level file, write in ./src/hey.rs

// Declare that there is a module "stuff" ...
mod stuff;

use std::env;

// Use the "hello" submodule from the declared "stuff" module
use stuff::hello;


fn get_arguments() -> Vec<String> {
    // Get the arguments from CLI and do check, then
    // return the arguments, without the program name
    let args : Vec<String> = env::args().collect();

    if args.len() <= 1 {
        eprintln!("Please specify arguments !");
        std::process::exit(1);
    }

    // We get a slice of an existing vector
    return args[1..]
        // but to return it, we need it to be a vector - so cast it back explicitly
        .to_vec();
}


fn main() {
    let args : Vec<String> = get_arguments();

    for name in args.iter() {
        println!("{}", hello::greet(name) );
    }
}

Then write a src/stuff.rs file

// This is the side-module "stuff"
// It has submodule implementations in "stuff/"
//   declare their existence here. Note that this is
//   needed even for each submodule to see each other
pub mod hello;

Then write the submodule in src/stuff/hello.rs

pub fn greet(name:&str) -> String {
    // Create a "mutable" and "owned"/heap String from the string-slice (native, on the stack)
    let mut phrase:String = String::from("Hello, ");

    // Push the stack content from "name" into the head object "phrase"
    phrase.push_str(name);

    return phrase;
}

Build it, and run it

rustc src/hey.rs

./hey Alex Jay

🔝

Lua

Lua is a popular tool scripting language - it's used in a variety of settings notably Roblox, the GNU Image Mainpulation Program ("GIMP"), and if I may just momentarily shill, in Minetest. I used to write and maintain various mods for Minetest, and it's even fair to say I did a lot of my learning during that time... there's a lot of learning that can happen on spare time when you're also having fun 😄

Any application that does embed Lua likely embeds a specific reference version of it, and may add/remove library features as suits the application. For this test I installed the CLI interpreter for Lua 5.4, but any version of Lua would work though with this example.

Most notably though, I bring attention to the fact that top-level symbols are global by default, and that to namespace a function, you need to declare it on an existing variable.

In ./hey.lua

-- Executes a file , any non-local symbols enter the global space
dofile("stuff/hello.lua")

if #arg == 0 then
    print("Please supply arguments")
    os.exit(1)
end

for _,name in ipairs(arg) do
    phrase = stuff_hello.greet(name)
    print(phrase)
end

And in ./stuff/hello.lua

-- This is a global variable that will go into the global space
stuff_hello = {}

-- This implements a method onto the stuff_hello object/table
--   for namespacing
function stuff_hello.greet(name)
    return "Hello, "..name.."!"
end

And run

lua hey.lua Alex Jay Pat

🔝

Java

We love to rant about how verbose Java is, but having rattled through a few of the other languages now, it should be possible to see how comparing simple "Hello World" examples does it undue disservice.

Yet there are still some funky constraints. All files should be in a top-level package directory, and your current working directory should be its parent. Every file contains a public class, and each public class must be in a file with an exact corresponding name.

In helloyall/Hey.java write:

package helloyall;

import helloyall.stuff.Hello;


public class Hey {
    public static void main(String[] args) {
        Hello greeter = new Hello();

        // Actual arguments start from zero - program name is
        //   not stored in the args array
        if(args.length < 1) {
            System.out.println("Please supply arguments");
            return;
        }

        for(int i = 0; i < args.length; i++) {
            String phrase = greeter.hello(args[i]);
            System.out.println(phrase);
        }
    }
}

And in helloyall/stuff/Hello.java write:

package helloyall.stuff;

public class Hello {
    public String hello(String name) {
        return "Hello "+name;
    }
}

Compile and run:

# So that java knows where to import from, CLASSPATH needs to also include local project directory
export CLASSPATH=".;$CLASSPATH"

# Compile
javac helloyall/Hey.java

# Run - uses package notation
java helloyall.Hey Jay Alex Sam

🔝

Groovy for Jenkinsfile

This is one that stumped me for ages - I had a right rant at Jenkinsfile Groovy a few years back for just how hard it was to find decent concise information about it.

Groovy is a language designed to be a "scripting language" that runs on the JVM, which means some of its actual representations are heavily influenced by Java's ~~limitations~~ requirements.

One of the key concepts to retain first is that the top level Jenkinsfile code itself runs on the server. A bad Jenkinsfile can crash a Jenkins server, so don't let just anyone write Jenkinsfile code to run on your server. note then that any SCM checkout should always be done on a node.

Another corollary to this is that each statement is passed on from the main server to the node for execution - as in, the main server steps through the Jenkinsfile, and dispatches steps individually over to the execution node. As a corollary to this corollary (get comfortable with deep-nesting in Groovy!), the for and while loops are forbidden, in favour of iterators.

A second key concept is to know that every Groovy script is a class with instance methods. This has some very interesting effects which I have touched on before and if you are about to embark on such a journey, may I humbly suggest you read those notes. They are to be essential to getting your head around some oddities you will surely encounter.

Declare your main jenkinsfile at the root of your repo, write ./hey.Jenkinsfile

node("runner-node") {
    // "runner-node" should be an agent or label defined on your Jenkins server
    // Try to always confine code to a node that is _not_ your master node.

    // The hey.Jenkinsfile should be in a git repo !! Or SCM of your choice
    // The files must be checked out to a node workspace first
    checkout scm;

    // We can load from the workspace, assuming your files are at the root of the repo
    hello = load "${env.WORKSPACE}/stuff/hello.groovy"

    names = env.NAMES.split(",")

    // Iteration - not a loop ! - with a closure
    names.each { name ->
        phrase = hello.greet(name);
        println(phrase)
    }
}

From the root of your repo, write stuff/hello.groovy

def greet(name) {
    return "Hello, ${name.trim()}!"
}

// The whole file gets encapsulated to a class
// and any "naked code" goes into a static run() method of the class.
// We must `return this;` so that the `load` operation receives the loaded instance.
return this;

Commit this code and push it. On Jenkins, create a Pipeline job, and point it appropriately to your script. Make it configurable, and give it a String parameter, into which to place names. Save and choose to "Build with parameters" - you can enter the names now, like Alex, Jay, Sam and see the result on the console output.

🔝

Perl

My search for how to do this turned out to be unexpectedly infuriating. Some many years ago, I did do a job which involved integrating our Perl solution with customer environments, and I do not remember it being this maddening...

Searching the web, I get a bunch of results as usual for using other peoples' modules, but nothing for organising my own code.

StackOverflow tells me to RTFM, but the FM is so jargonic that it might as well be an implementation spec for the feature itself rather than a user manual. An attempted answer simply doesn't seem to work, bailing with Can't locate A/Calc.pm in @INC , and same with a simpler answer on StackOverflow . It pointed also to more documentation perlmod. Some blogger AlivnA gives a pointer which helps get past the @INC issue, but the subroutine still doesn't seem to export, and at this point I've given up.

This is just a sample of the hair-pulling confusion trying to hunt down how to import a file from a subfolder. I begin to suspect that this even even possible/a done thing... Perlists, please prove me wrong, I beg of you...

EDIT: The gods have answered, or at least, my friend @pozorvlak took pity on my plight and put together a working Hello Y'All Perl example . The incantations give rise.

I ended up with this 🛑 non working code.

In ./hey.pl I have

# Add to @INC list
use lib './stuff';

# Import from anywhere in the @INC list
use hello;

# Results in "undefined subroutine &hello::greet"
my $phrase = hello::greet("Jay");
printf("$phrase\n");

and in ./stuff/hello.pm I have

package stuff::hello;

our sub greet {
    my $name = shift(@_);
    return "Howdy, $name!";
}

1;

And I run with

perl hey.pl

# Undefined subroutine &hello::greet called at hey.pl line 4.

How is it so difficult to do this supposedly extremely commonplace activity when the OCR of splattered paint resolves to runnable perl 93% of the time ???

🔝

Bash for Bash Builder

If you'll indulge me, I've kept this one til last. I wrote this tool during a time I had shell scripting as my main language, being more sysadmin-oriented. One thing I wanted was to be able to write various scripts, but with re-usable code, instead of copy-pasting it around.

source-ing code is unreliable - it depends on the user's current working directory, and sourced files do not have the benefit of knowing where they themselves are - $0 only ever hints to the top level script's location. It was important to be able to perform imports from known absolute locations.

As such, exporting BBPATH=/path/to/bash/libs is important, and taken care of by the installer into the ~/.bashrc file.

With that done, write a ./hey.sh:

# The out.sh lib allows doing colourful output
#%include std/out.sh

#%include ./stuff/hello.sh

if [[ -e "$1" ]]; then
    out:error "No argument set"
    exit 1
fi

# Did you know that ":" is a legitimate character for a bash function name?
# Any character that can be a file character can be used.
# Used here for namespacing.
phrase="$(hello:greet "$1")"
out:info "$phrase"

And also write a ./stuff/hello.sh file:

# Bash-Builder provides some syntax sugar macros for putting names on
#  positional arguments. The rest remain in the "$@" array variable
$%function hello:greet(name) {
    echo "$name"
}

Build it and run it:

bbuild hey.sh bin/hey

bash bin/hey

Note that even this works, due to the imports being resolved at build time:

cd stuff
bash ../bin/hey

Changing directory and calling like that does not work with regular bash sourcing. Sourcing was one of the major issues I had started developing bash-builder to solve. Once I had the macros for the inclusion work, it was only a small step to include the syntactic sugar macros of %function(name1 name2 ...)

Arguably, a lot of the richness comes from its associated project bash-libs which provides a sort of standard library for extra goodiness ...

🔝

Conclusion

There we are. A whirlwind tour of a dozen languages demonstrating a Hello-World that's a bit more useful than the basic.

If you write language documentation and tutorials, I hope this inspires you to put something like this in the introduction or first chapter, to give a better flavour of the language.

Any popular languages not on this list ? What does your list of languages look like implementing "Hello Y'All"

Baby steps with Go

TaiKedz — Fri, 02 Aug 2024 14:49:06 +0000

Image (C) Tai Kedzierski

I decided to give Go a try on my journey to pick up a new language that would be useful to my career and interests. This time I've been having a go at Go. I think as first impressions go, it's pretty nice.

This is not a guided tour, and arguably, not written for anyone else other than myself, as some personal reminders.

I gave myself a small project for it called Os-Release-Q . My intention was to be able to have a binary on any system I manage, such that I can print out exactly the information I need, without needing to parse or eye-grep for it.

First hurdle : `import`

Searching the web talks a lot about importing other people's packages , but very little about organising one's own code. Even the docs focus on go get rather than separation of concerns.

I encounter this hurdle quite a bit in every language, as each has its own idiosyncratic philosophy on how to go about it, and what limitations each has or imposes. (I wrote a piece on this as a direct follow-up to learning Go's import idisyncracies)

Of all the activities I undertook in learning the basics, coming from a predominantly python background, splitting my code into multiple files was what took me the longest to get answers to. In summary, I found the following:

top level needs a go.mod declaring module module-name
I can then set a src/ directory at top level, and a src/main.go in which to place my main function , with a package main declaration at the top
putting code in other files is a simple as creating a file like src/others.go with a package main declaration.
All functions and variables become available directly in any other file of package main , but the files need to be explicitly stated on the go build FILES call - e.g. go build src/main.go src/others.go

For local submodules, the submodule must reside in a folder. It can declare a package submodule-name .

Say it is in src/submod/, with main implementor in src/submod/submod.go. In main.go we do import "module-name/src/submod" (with module-name pulled from go.mod). And then we can call submod.SomeFunction().

We note that submodule functions are only available to importers if their name starts with a Capitalised letter. So no doing submod.myFunction() - it has to be submod.MyFunction().

There are surely other considerations around submodules and imports, but as far as keeping code organised and segregated, this is the essentials.

To keep things sane, I tempted to only have one file declaring package main, and isolating the rest into submodules - these get imported automatically without needing to be declared in the go build FILES list of files.

Doing basic tasks

After I had resolved this specificity of Go, the rest fell in to place quite easily. For every basic task there was of course a StackOverflow entry, or a GoByExample.com page, and more basically, the Go language reference.

String handling is done via the strings package
Array handling has a number of native functions, of which the base_array = append(base_array, item1, item2) pattern - it also works for extending an array with the values of another via append(base, other_array...)
Error handling is done by passing out error objects typically, but not necessarily.
a "log" lib exists for a handy pre-configured no-faffing log. It includes a log.Fatal(message) call which logs an error, as well as immediately exiting.
Calling subprocesses is easy via the "os/exec" library, using exec.Command(base, args...) pattern

Compile-time errors were by and large intuitive, based on a basic understanding of the language (it's not the first time I looked at it, and on the sruface, it's a very simple and straightforward syntax)

Two particularly common tasks deserve their own paragraphs.

Error handling

Basic error handling is often commented as being cumbersome, literally needing to handle errors in the midst of control flow. This may be anathema to programmers coming from a try/catch workflow, but handling the issue at the point where it can happen isn't so bad.

// explicit return item `err` forces us to be aware of it
// but having the ability to check it in the same breath is not so bad
if result, err := someCall(); err != nil {
    log.Fatal("Sorry.")
}

fmt.Println(result)

Compare try/catch way

try:
    result = someCall()
except:
    print_exit(1, "Sorry")

print(result)

I've seen hintings at people being unhappy with the whole "got to check err in the middle of my code" which certainly removes the ability to group-handle errors. For example in Python:

try:
    res = do_thing()
    res = and_more(res)
    res = and_other(res)
except ErrType as e:
    print(e)
    exit(1)

If ErrType is in fact thrown by all three , this is a handy way to catch each scenario. In Go, it's a little bit more cumbersome

func do_all_the_things() {
    if res, err := do_thing(); err != nil {
        return err
    }
    if res, err := and_more(res); err != nil {
        return err
    }
    if res, err := and_other(res); err != nil {
        return err
    }

    return nil

func main() {
    if err := do_all_the_things(); err != nil {
        log.Fatal("Sorry")
    }
}

This is very much predicated on long sequences of calls wanting to be grouped under a same error handling routine, and how much that actually happens to people I cannot say. I guess I'll find out in my own time, but I remain, for now, unfazed.

Argument Parsing

I can't help but feel that the implementation of the flags library is a bit half-baked. Evidently people are used to and OK with it, given its survival in its current form.

Calling program -flag arg1 arg2 gives us the toggle that flag is set up to do, and positionals := flags.Args() returns us the array of ["arg1", "arg2"]

However calling program arg1 arg2 -flag does not toggle whatever -flags is supposed to do, and instead gives is positionals as ["arg1", "arg2", "-flag"] wherein the flag was not parsed.

This may be useful for passing in a sub-call like program colorize ls -l where the ls -l is passed down literally - so I can see a use case.

It's just that most programs out there allow flag arguments anywhere around positional items. ls dir1/ -l dir2/ is the same as ls -l dir1/ dir2/, and this is a convention that holds on the vast majority of Unix and Linux commands.

It may be just that this is something to get used to - and worth calling out.

EDIT: In the end, I implemented a parsing lib of my own with the niceties that I desired

Purpose and use case of Go

The file import paradigm aside, I found it pretty easy to get my basic application implemented. Anything I did wrong felt fairly obvious and the errors were meaningful. It really does feel like I can just focus on "getting things done."

From my very meagre amount of use so far, and taking my specific needs into account, I can see the following advantages

easy to get started
compiled binary, no runtime dependency
simple language with types is a step up from shell scripting
allegedly easy multiprocessing support

I thought having sparse types instead of objects and inheritance would be a hinderance, but so far so good. I get by without them in other languages, so I suppose when I do get around to defining interfaces and types, it will feel like a step up from Lua and bash. I hope.

One of the reasons I wanted to explore a compiled-to-native language was to be able to produce binaries that could easily be shunted around, without needing to rely on a particular version of a runtime being present.

A colleague once walked up to my desk in dismay, trying to solve getting Java 17 onto an old Node base image which was based on Debian 10 . Either he'd have to upgrade the Node version to get a newer base image, use a new Debian base image and install and configure Node manually, or scour the internet for a custom repo hosted by goodness-knows-who for a goodness-knows-if-hacked Java 17 that would run on Debian 10. Each solution would have ended up entailing further refactoring of the code base.

How much easier if the deployed software had no such conflicting runtime dependencies...

From an ops point of view, the one big gain I am feeling I would stand to feel is: I can easily write code, and build an ELF binary to then deploy on "arbitrary system X" and not have to contend with ensuring the right version of a given runtime is in place, and managing conflicting dependencies.

I've seen it touted before as an alternative glue language to replace shell scripting, and I can just about see that. One of the big advantages of shell scripting is the ability to pipe outputs from one command directly into another - but most often, the things that sit in that pipe are sed, grep, awk and other text-massaging utilities. These can be replaced by language-native functions, and my basic experience here with getting output from a command has been pretty good. The only impediment in this scenario is that the Go program needs to be compiled whereas the shell can just run. It really will depend on what the deployment requirements are, but as soon as "runtime dependency" kicks in, I would be fairly tempted to reach for Go.

I'm sure there are other benefits, and I have heard a great deal said about the ease of use of multithreading and multiprocessing in Go, and I do intend on cooking up a mini project to explore that as a next step - probably something that might listen for inputs on multiple channels, and perform some basic tasks in response. I have had a use-case for that in some test automation tasks I've had before, so it's not alien to me at this point.

Rust's Option type... in Python

TaiKedz — Fri, 13 Oct 2023 14:14:18 +0000

Cover Image (C) Tai Kedzierski

How many times have you written/seen code like this:

data = get_data()
print(data.strip())

Looks reasonable right?

How about if data == None?

It is so easy to assume thingness and forget to handle nullness. So frequent. In Python, Java, JavaScript, and many other languages, this is prone to happening.

Then some languages, like Rust, have a semantic way of dealing with this... and I pondered, what if we tried to intorduce this to Python? How would it look?

Just for a laugh, and in between waiting for integration tests to finish running, I threw this together: a quick Python implementation to mimic Rust's Option type

In Rust, the Option enumeration type is used with the match operator to force handling the concept of null-ness, without treating it as a first-order value, unlike in languages that have a null representation.

Does that sound weird? Let me digress a moment

Nullity as a typed value, not a stand-in

In languages where there can be a None or null placeholder-concept, it works like this:

functionA() returns a value, say of type String (or other, it is immaterial). If functionA() returns null, it effectively says it returned null instead of a String. Trying to treat this null as String (by calling its methods, concatenating it, etc) leads to errors.

In Rust, this cannot happen.

In Rust if functionA() says returns a String, it always returns a valid string. If we want to express that a non-valid value can be returned, then functionA() must flip: it returns an Option instead.

We must then handle the Option enumeration type - check if it is the variant known as None , or check if it is the variant known as Some.

We can then .unwrap() the result explicitly, or through pattern matching implicitly:

# Panics if we get a None variant
let value = functionA().unwrap();

# vs

match functionA() {
    None => println!("Fail!"),
    Some(value) => println!("Success: {}", value),
}

In Python

So how does this experiment look in Python ?

Well, we're not replacing the native None type. We'll use a Null class (a regular class), subclassed to Option. I didn't include a Some type, but it would be simple enough - just class Some(Option): pass and it's done. Everything is in fact implemented on Option

We are then able to wrap this around any return result we care to add to our script to make it return an Option.

def some_func() -> Option:
    some_value = None

    # ... implementation ...

    if valid:
        return Option("some value")

    elif not_valid:
        # Explicit null-return
        return Null

    # or even
    return Option(some_value)
    # (will behave like Null if some_value is still None)

res = some_func()
if res.is_null():
    ... # handle it
value = res.unwrap()

This forces whoever uses some_function() to consider that nullity can come of it, before gaining access to the value itself:

# Without a check, this may "panic" (borrowing from rust's terminology)
#  the program exits systematically with an error message
value = some_func().unwrap("failed to get data from some_func")

# On occurrence of nullity, a default value is substituted instead.
value = some_func().or_default("default data")

This makes for explicit force-handling of None , where before it could easily pass under the review radar silently.

An interesting side-effect is that managing default-substitution is now done outside of some_func's logic; the design of some_func can stay unencumbered with specifically dealing with alternate eventualities, just as the caller code can declare the default value without multiple lines of if/else checking.

We also don't have a match keyword to use, but this doesn't impede the usability in any significant way - merely having to acknowledge unwrapping brings gains, and the or_raise() method allows some additional flow control.

Should this even be used in Python ?

I am actually tempted to use it in my projects... as a way of forcing awareness of null possibility.

It ensures that nullness cannot be ignored/forgotten by the caller, promoting more conscientious coding, and helping avoid some common pitfalls.

It is succinct enough to allow minimal changes in code flow, but explicit so that it cannot be ignored.

But it's very un-idiomatic, literally "trying to write Rust style code in Python."

And yet.

🪙 A penny for you thoughts?

Over to Rust - Gotchas for Beginners

TaiKedz — Sun, 10 Sep 2023 16:38:42 +0000

Cover image (C) Tai Kedzierski

I had an idea for a simple utility the other day - a way to manage a *nix $PATH. At it's core, it has one file reading operation, which also does some line filtering, and one file-append operation. That's all.

I wrote it up quickly in python and threw it into a Github gist.

I then realised, hey, this program is pretty simple. I wonder how easy it would be to write it in rust? And so the journey began...

Rust-lust

What is Rust? Articles abound on the Net and this site. Suffice to say it is a new language that has attracted the envy of many - StackOverflow blog posted some thoughts in 2020, and in the 2023 developer survey, over 80% of respondents who have used it in the past year want to keep on using it.

I've been wanting to properly sit down and learn rust for a while now - at least a couple of years. I mainly work with python, which is a superb language for getting things done fast - but that low barrier to entry encourages one undesirable thing: messy, buggy code. Its surge in popularity has been attributed plenty of times to it being easy to learn, the dynamic-typing and very forgiving runtime allow you to see results in minutes, sometimes even seconds, but at the cost of letting you get away with code-murder.

I really want to learn rust not only for its speed and memory safety (I am perennially eyeing up the goal of writing a multiplayer game server), but also to benefit from thinking about code fundamentally differently, and open up new practices to apply further afield.

The learning curve is proving to be not so gentle of course, but its advertised advantages feel worth it. But as all things in learning, you need some practical exercising to properly internalise the lesson.

Some resource I had been following suggested first reading through The Book without typing any code, and then re-visit the book again after ingesting the basics with a more holistic idea. I had already done that first part (I couldn't help myself, I was experimenting as I went along), and was now itching for an actual project. The Book provides some useful mini-projects to try out by way of a practical tutorial, but what I really wanted was something I would find useful myself. pathctl was just what I needed. There were a few very simple requirements, and I had proven its simplicity by doing it in a language I know.

After an initial fortnight of dipping into The Book, it has taken me one week (first python on Monday, basic completion of rust project on Sunday).

Here's some notes from my mini-journey so far - things that The Book didn't cover quite so obviously, or where I was just thrown as a python/dynamic/other-language programmer.

Ownership and the Borrow Checker

I was exposed to some C programming many years ago at university, and I remember distinctly disliking having to juggle heap pointers and manage memory de/allocations. They're not a difficult concept to grasp at all, but managing the wee blighters is a chore - testament to which the numerous dereference bugs, and the very existence of rust.

Rust's Ownership concept and the compile-time Borrow Checker feel jarring to work with at first, especially coming from a dynamic language, but its novelty fades away very quickly into a level of semi-comfort.

Operations on Strings usually return &str slices , which are references to the original data - owned. So if you have a vector (list for the pythonistas out there) of items, and you want to return a vector of modified strings, you need to explicitly convert these to new objects. This might seem memory-inefficient at first, but that's in fact what pretty mcuh any language will do under the hood. It's just explicit in rust:

fn load_valid_lines(path_str:&str) -> Vec<String> {
    // The actual file opening/reading etc is done elsewhere
    read_lines(path_str)
        .iter() // Explicitly iterate

        // trim() returns a &str , referencing the iterated line
        // which cannot be returned (owned and remaining in the function)
        // so we produce a new String using a one-line closure
        .map(|line| String::from(line.trim()))

        // the `is_valid_line()` function checks the item without taking ownership: pass in `&line`
        .filter(|line| is_valid_line(&line))

        // and finally create a vector, from the map .
        // Note the "::" and the doubled '<' for generic-in-generic
        .collect::<Vec<String>>()
}

Enums

In the two languages I had previously looked at for enums, python and Java, their enum types were both cemented collections of values.

Rust does it a bit differently - additionally to being a list of fxied values, enums can also encode for a simple type family using value tuples.

enum IPAddress {
    IPv4(u8, u8, u8, u8),
    IPv6(String)
}

fn main() {
    let local = IPAddress::IPv4(127,0,0,1);
}

This also means that Enums can have shared implementation functions ... and perhaps some more.

I gave up on using Enums for my argument action logic, because I got myself into a bit of a tizzy in trying to use them.

One use-case of Enums however ties in with the match mechanism, native to the language, which allows thinking of returned values as being of particular distinct eventualites:

Result Enum expresses the idea that an operation could have succeeded or failed, and forces the caller to handle both
Options Enum expresses the idea that an operation could have produced Some(T) result, or None (an eventuality if you will, but not a value)

There is no `null` only nullity.

A special Enum Option allows handling the cases of None or Some(T) - there is no null or None type or value that can be passed around, but instead the Option enumeration that allows expressing that eventuality, and handling it over the match mechanism.

Instead of being able to say "I've got nothing", with ambiguity over whether NothingTM might be actually a new brand of egregiously air-filled popped snacks; rust forces you to conceive that "I didn't get any valid value from the operation."

It's a very subtle difference, but it does introduce a deliberate idiomatic way of thinking about the concept.

Error handling

Error handling is also done frequently via the match item on operations that return Enums. A common Enum is Result which can either be Err or Ok.

You usually need to handle it with your result to get the handler you want:

match FileOpener::open(file_name) {
  Err(e) => { eprintln!("Failed: {}", e); },

  Ok(file_handler) => {
    println!("I'm in!");
  }
}

Oftentimes, code snippets will use unwrap() on their snippets for the sake of brevity, but that removes control from you the programmer to simply panicking the process, and exposing programming guts at runtime:

// Panics altogether if something goes wrong
let file_handler = FileOpener::open(file_name).unwrap();
println!("I'm in!");

So whenever there's an unwrap() in the chain, it's better to split it out into a match and deal with it.

-    read_to_string(path_str) 
-        .unwrap()           // panic on possible file-reading errors
-        .lines()            // split the string into an iterator of string slices
-        .map(String::from)  // make each slice into a string
-        .collect()          // gather them together into a vector
+    match read_to_string(path_str) {
+        Err(e) => {
+            eprintln!("Error reading file '{}': {}", path_str, e);
+            std::process::exit(1);
+        }
+        Ok(data) => {
+            data.lines()            // split the string into an iterator of string slices
+                .map(String::from)  // make each slice into an owned string
+                .collect()          // gather them together into a vector
+        }
+    }

"Catching" Errors

On that note - there are two kinds of errors to consider:

the kind you predict can happen and have coded around
the kind that you really shouldn't have let happen (panic)

(there are of course the kind you couldn't conceive of ahead of time, but we'll let that one slide)

I've seen it posited that any error that you can identify in Result type or by simply returning a given value can and probably should be handled. If you don't handle them (naughty use of unwrap()?) and allow the program to panic as a result, it's on you. Handle them there and then, and if throwing them up a level, do so using idiomatic structures like the Result and Option Enums.

Conversely, if a program panics don't seek to catch that. Relying on the program panicking as a way of doing flow control is essentially viewed as sloppy - something you didn't care to cleanly pass values for, showing that perhaps you didn't quite think of that.

(Of course, if a library deep-down chooses to panic rather than return a Result::Err, it's up for debate. I'm not yet conversant enough in the ecosystem to know what should be thought of with such an eventuality)

The use of try/catch in many languages as a control flow mechanism is seen as encouraging the programmer to see an exceptional circumstance as normal possibility. It might be semantics, it might simply just be idiomatic.

But rust very pointedly encourages the programmer handle cases properly, and see a panic as a symptom of something arising from improper design.

EDIT - I've been around a few more doc pages and modules, and found that my above understanding needs to be more nuanced... certainly using .unwrap_* alternative functions can offer some level of acceptable blissful ignorance.

In The Book itself, there is a section dedicated to understanding when panicking should be considered suitable.

It can be seen that ultimately, if there is really no sense in keeping on, it might be sensible enough to just panic.

My own take however is that if you choose to panic, the only result is the aborting of the program - and often that can lead to unintended consequences. Consider:

fn collate_files(in_files: &Vec<&str>, connection: ConnectionType) {
    // Create a temp file with each 'X' replaced by an arbitrary letter
    let output_file = tempfiles::new(".temp-XXXX");
    for input_file in in_files.iter() {
        process_and_write(input_file, &output_file); // panics?
    }
    connection.send_file(output_file);
    output_file.remove();
}

If you write process_and_write such that it panics if the input file cannot be found (for example), not only do you skip writing the result back over the network, but you also prevent any cleanup actions.

Especially, if you write a process_and_write function as a library utility for others to consume, you prevent others from performing their own cleanups as needed. So as best possible, never panic when you are writing library. By the same token, don't depend on library functions that panic when providing your own library.

Modules and submodules

Rust distinguishes (though I found the documentation a little light here) modules and submodules, and the programmer needs to declare them properly.

Say I have a module util.rs , sidecar to my main.rs. Say I want my util module to have two submodules, slice.rs and dice.rs.

In my main script, I can call

mod util;

And that tells the compiler we expect a module util.rs to be present (there's a legacy variation for this, but this is a simpler way for me to brain this). Inside that file, I must use

mod slice;
mod dice;

This is not (only) for exposing those submodules to my main file. They actually tell the compiler that those submodules are expected to exist.

Even if my main file only will use slice , if I want to access some functions from dice , the latter must be declared in the supermodule.

Binary sizes

A normally rustc compiled/built rust program is bundled with a load of debug symbols by default, and hello_world typically clocks in a 12 MB . This is huge, given that this is supposed to be a systems language, one of the applications for which is supporting embedded systems.

Running rustc -C strip=symbols mycode.rs removes these, paring the binary down to a much more reasonable 331 KB.

In the Cargo.toml file, this can be cemented as

[profile.dev]
strip = "symbols"

[profile.release]
strip = "symbols"

The systematic presence of debug symbols seems to surprise most people - I presume that in other languages, the default behaviour is to not do this, and include them when the compiler is specifically requested to do so.

I suspect this novel default behaviour is what allows new rust programmers to have debug information available out of the box. Removing the debug symbols amounting to explicitly "removing the training wheels", where as in the opposite case, training wheels need to be found. I dunno, just a hunch 🤷

Is it easy ?

I often hear about how "Spanish is easy to learn, Polish (etc) is difficult," and of course, that really depends on where you're starting from. If you know English, it is similar enough to Spanish that it's easy to pick up, relative to the case-declensions and non-latinate roots Polish. But to a native Czech, the complete opposite is likely to be true.

So how is rust's learning curve?

I can spot Python code written by a C programmer a mile off. Learning a new language in its idiomatic form will always be a challenge. Don't just try to "write like (my language X) in Rust." You won't get the benefits. With that in mind:

For someone used to JavaScript or Python as their main, rust stands to be quite involved. If you started your career purely mucking in with dynamically-typed, object-oriented, exception-catching languages without doing any lower-level programming introductions, it'll be quite an alien world.

Strict typing, explicit heap references, strict ownership and a fussy compiler all combine to make it feel like progress in learning is slow.

In my case, I did come to rust with some exposure, an era and an age ago, to the concepts of pointers and stack/heap memory, and other odds and ends, so I've managed to dust off the old knowledge to just about follow, but I'm not completely in my element either.

For a C programmer, there are idiomatic challenges around the match/Enum techniques and perhaps others, and there are types. You can't just declare everything as a pointer and call it a day.

I expect in the end the ones who will have it easiest are those who work both with C and with a dynamic language on a frequent basis, going to and from each other, and comfortable with the idiosyncracies and idiomatics of each.

Thankfully, it's not Haskell-levels of maddening.

Considering Syncthing at work

TaiKedz — Wed, 23 Aug 2023 08:08:35 +0000

Header image (C) Tai Kedzierski

We had a need at work to pass large files around, to colleagues who could not get onto the VPN. It was company data, so spinning up an instance on a cloud provider was not an option. We needed something that would either go through company servers, or a peer-to-peer encrypted transfer. Enter: Syncthing.

Syncthing uses end-to-end encryption to send files directly to other devices. If two devices are on the same network, the instances communicate directly with eachother. If not, they send traffic, encrypted, via a relay server (akin to a router).

This solution can be useful for sending large files between workstations, when other means are unavailable.

Of course, the Internet is vast, and the bits and bytes need to make a few hops before they can reach destination. It's also an external solution, so your company might have questions it would want to ask before greenlighting it.

Here is my reasoning that allows me to say, "hey, under these conditions, this solution is trustable."

Security considerations

Trustability of Solution

Syncthing claims to be secure and E2E encrypted. The solution is well-known and several years old ; it is reasonable to presume it has attracted sufficient scrutiny from the security community and nothing has been flagged as troublesome/buggy that still exists from a security standpoint.

For most purposes, including corporate use, I would tend to be satisfied with the general status of this tool.

I would not use it in a highly sensitive environment without further scrutiny, and proper approval channels.

Trustability of Build

Use of syncthing in sensitive environments may need to consider how to determine whether the build of the syncthing binary is untampered with.

Installing from a well-known distro's default repositories provides a usually suitable level of assurance that it will abide by its security claims, and that the implementation is pristine (within the scope of the distro repo's practises).

I would assert that any of the mirrors for Debian, Ubuntu, Fedora, RHEL and SuSE are suitable in this regard.

Trustability of Relay

If two syncthing instances are on the same network, they will transfer data between eachother directly, without a third-party. If the two instances are on different networks, they will make use of a publicly hosted relay machine.

All content being E2E encrypted, this should not be an issue whatsoever for security. This does depend on the installed copy being clean, and the caveat in "Trustability of Solution" stands.

Beyond that, the relay acts much like a router - packets pass through it, fully encrypted.

Installation

(On Windows, use WSL with any of Ubuntu recent LTS, Fedora recent, openSUSE LTS, Debian recent...)

The Syncthing team provide their own APT repository - instructions for adding it are on their own website: https://apt.syncthing.net/

Do NOT use user-contributed repositories:

ppa - there isn't an official one, use the above APT instructions
snap/flatpak sources - don't do it
docker images - don't do it
third-party repositories - don't do it
Arch AUR is to be rejected outright for any privacy/security-related solution

sudo apt-get update && sudo apt-get install syncthing -y
# Run it in the background
syncthing > syncthing.log 2>syncthing-errors.log &

Syncthing will open a browser to your http://localhost:8384

Add a device by going to the bottom right section and "add device"

Device (example): A24RVJV-RCBOM6L-6PRUIEX-YRDRD6C-LWFFS7C-3Q2XN8G-IC5ZFWG-R2WBCQ2

You can get the ID of your own device by going to top right menu: Actions : Show ID. This ID can be shared with others. This allows a device to be recognised by a peer.

The operator of a device can then choose to share a specific folder with another device it has recognised and registered, it will send a share request to that device. The device receiving the share request must confirm the share.

With this share established, the workstation operators can use the defined sync folder to synchronise the data

What is DevOps, Really?

TaiKedz — Tue, 22 Aug 2023 16:11:03 +0000

Header image (C) Tai Kedzierski

What is DevOps, really? Straight up: no tool "is" a DevOps-enabling tool, in-and-of-itself.

Various buzzwords containing "DevOps tooling" and "Continuous X" have sprung into being, and whilst they are useful as conceptual terminology, reaching too soon for their corresponding marketing productizations is to leap-frog over the essential ground work of defining what effect does it have on our practices?

In its most abstract form, "DevOps" has been expressed as being "developers and operations working together."

The term comes broadly from the web-oriented/Web 2.0/SaaS space where developers would make change to the core product code, and operations would take these in whatever form and deploy into production, following a basic workflow of production-then-deployment. Observing the resulting bottlenecks, and applying Agile principles of small, frequent iterations, the term DevOps eventually emerged to describe a practice of bringing bothe developer and operations into the activity of refining release cycles. Over the years since the coining of the term, various interpretations have arisen.

A question to keep in mind whilst considering options and the below notes is to ask why, or: "what are we setting out to achieve by implementing a DevOps approach?"

faster delivery of product
shorter development cycles
easier maintenance of release branches
more maintainable release procedures
better auditability of builds (easy to identify what changes went where, and "where to get the latest X")
more automation, less manual operation
better synchronicity and coordination of deliverables (builds, documentation, release notes, …)
... all of the above, and more ...?

(if these sound familiar, it is no accident)

Interpretations of "DevOps" ➿

One issue has been a focus on defining DevOps as being a set of tools rather than a way of working, and different organisations approach the domain differently, depending on those two ways of thinking.

For the "set of tools" way of thinking, see the myriad Solutions produced to "do DevOps better."

For a "way of working," this is up to the organisation to define, but ultimately boils down to

first collectively (all members on both sides) agreeing on the architecture and flow and goal, to get from "source code" to "deliverable"
then deciding what tooling meets that design

As with many things in software engineering, "do one thing and do it well" is a good starting point. It is not an inviolable rule (the primary doctrine is to avoid doctrine), but there should be good reason to introduce complexity inherently into a given system.

Often, when organisations start on a DevOps route, tools are invoked, installed, and setup, and things are strung together ad-hoc. Whilst this can work it usually causes a lot of 🍝 tight-coupling where none should exist, and multiple-self-referential 🪢 knots to weave into the codebase, making for very difficult technical debt to cement itself (mixed metaphors be pardoned).

Deliverables 📦

DevOps has gained wider-spread adoption into non-web development organisations - the end deliverable being not "a deployed website," but some built binary which can then be shipped to another team or process, or indeed to a customer. "To publish" can mean any of

to push the files to a web server (original scenario)
to put the final tarball in a artifact server
to hand a ZIP file download URL to the manufacturing division for installation on devices.
to produce a DEB or RPM ready for clients to pull
to place a build output on a filesystem (ready for another process to consume)

Essentially, determine: where do our pipes end? Deliver to there.

Dev and Ops working together

In my view, the main aim to thinking about the problem up-front is to ensure good interfacing between teams' responsibilities, and segregating the responsibilities of individual scripts/pipelines. Each script should either:

define an environment (a setup script)
- the deliverable is an environment itself
execute a single build (a build script)
- consume source code and/or dependencies, in the context of an environment
- the deliverable is a build binary
orchestrate builds (a pipeline)
- define the order of builds
- inform later steps of outputs of earlier steps

This might sound like Dev and Ops are not working together - but quite to the contrary. To achieve this, they must work in good coordination. If there is not one single team, then there are at least two (usually more), with distinct capabilities and specialisations. "Working together" (in the interpretation I am musing here) means the team members can reach out to eachother across teams directly to achieve the agreed goal. The earlier statement of "first collectively agreeing" what the goal is, and what the scopes of responsibility are, is prerequisite to this.

Each team should be able to operate on its own responsibilities independently for routine maintenance without affecting eachother, but both should come together to agree on a number of points:

Dev needs to declare 📜 "this is how to invoke the build script in a minimal amount of commands"
- make clean && make all (for example)
Dev must define 💻 "these are the environment requirements" ("needs to be a Rocky Linux 9.1 server, with libcurses-dev and lxml-dev libraries installed, with gcc, ninja and jinja2 tool chains")
Ops needs to supply the 🌉 infrastructure and pipelines on which to execute the build and delivery mechanisms
Ops needs to provide any 📚 other outputs (like build logs, test reports, etc) as corollary deliverables, through desired channels - package repos, etc - and advise how to access these
Ops may also be directed to integrate 🔀 other operations to the pipeline - running tests written by automation QA; or producing corresponding user manual PDF/HTML/InDesign files
- A further inclusion of another 👋 Dev/Author team!

To achieve this, regular close collaboration is needed. None of what is "supplied" by either team is set in stone - particularly, either team will need adjustments from the other to achieve the goal.

Distinct Pipelines and Deliverables

A way of organising the flow of source-code-to-deliverable is:

🌱 Identify base component parts - build them and version them as deliverables themselves
🪴 Identify complex parts - consume base parts as one would consume a third-party library, and produce a new versioned deliverable
🌳 Identify a given product - consume base and complex parts as you would consume third-party solutions, and assemble these to produce a versioned deliverable: the software release.

Each of these comprise the following:

📂 A source of dependencies
- base component: a code repo
- complex components: a code repo, and a binaries repo for pre-built dependencies
- Product: usually pre-built dependencies, and some manifest files from a repo
🛠️ A pipeline
- each level has its own, stand-alone pipeline
📦 A delivery target (file format, and repository location)
- base and complex components: usually a package repository, but an artifacting server/filesystem is equally suitable
- product: usually a package repository, image repo (VMs and OCI/Docker images), cold-storage vault.

There are certainly more considerations than the above, but I wanted to highlight just how much of this is more than just tooling - you could probably put together a full, maintainable solution using just podman, nginx and a small-hundreds lines of scripting (and no, I am not setting a challenge to do this! but do feel free...).

It is about defining scope collectively, cultivating a shared understanding of the goal, and keeping components simple.

DEV Community: TaiKedz

I am so done here. The slop is overwhelming.

Shell/Python vs Go - practical thoughts form a reimplementation project

Alpacka, as a bash script

Shell scripting limitations

Why not python?

Enter Go

Package spec

How it went

Sub processes

Argument parsing

Publishing a Go module via hosted git repo

Yaml and Supply Chain Trust

Checking it on Alpine

Conclusion

Humanness at Work

What ((programming) language) should I learn this year, 2025 ?

💸 Cost of starting

💻 What Programming Language?

✂️ Narrowing Down

🔨 Mini-projects

📋 What languages for me this year?

🎉 Voila

Simple Python Logging - and a digression on dependencies, trust, and Copy/pasting code

Mini Logger Snippet

The mini-logger code

Learning Zig : a Python dev's learning notes

Zig? ⚡

Documentation and Learning Resources

Concepts

Arrays, Slices, Strings and ArrayLists

Stack and Heap

Allocators

Structs and Methods

Build System

Using other peoples' libs

Creating your own lib

Final thoughts

Beyond "Hello World" - comparing starter code in C, Rust, Python, TypeScript and more

Hello Y'All

Python

Bash

TypeScript via Bun

Web Server PHP

C

Go

Zig

Rust

Lua

Java

Groovy for Jenkinsfile

Perl

Bash for Bash Builder

Conclusion

Baby steps with Go

First hurdle : import

Doing basic tasks

Error handling

Argument Parsing

Purpose and use case of Go

Rust's Option type... in Python

Nullity as a typed value, not a stand-in

In Python

Should this even be used in Python ?

Over to Rust - Gotchas for Beginners

Rust-lust

Ownership and the Borrow Checker

Enums

There is no null only nullity.

Error handling

"Catching" Errors

Modules and submodules

Binary sizes

Is it easy ?

Considering Syncthing at work

Security considerations

Trustability of Solution

Trustability of Build

Trustability of Relay

Installation

First hurdle : `import`

There is no `null` only nullity.