DEV Community: Juan Julián Merelo Guervós

You can also have (some) type safety in JavaScript. Here's how.

Juan Julián Merelo Guervós — Mon, 10 Feb 2025 09:42:12 +0000

Destructuring for type safety in JavaScript

Juan Julián Merelo Guervós ・ Jul 25 '23

#javascript #typesafety #programmingpatterns

Writing extensions to git opens up lots of possibilities of enhancing teamwork through a commonly used tool. And #perl will make it easier.

Juan Julián Merelo Guervós — Mon, 10 Feb 2025 09:35:53 +0000

Writing git extensions in Perl

Juan Julián Merelo Guervós ・ Feb 10

#git #workflows #teamwork #perl

Writing git extensions in Perl

Juan Julián Merelo Guervós — Mon, 10 Feb 2025 09:34:43 +0000

Introduction

Most people will tell you git is a source control tool; some people will tell you that git is a content-addressable filesystem. It's all that, but the interesting thing is that it's a single-tool interface to frameworks that allow you to create products as a team.
Enter the absolutely simple extension mechanism that git has: write a n executable called git-xxx and git will dutifully call it when you make git xxx. Which is why, to make an easier onramp for students in my 7th-semester class in Computer Science, I created an extension called git iv (IV is the acronym for the class). The extension allows them to create branches with specific names, as well as upload those branches, without needing to remember specific git commands.

You might argue that remembering git commands is what students should do, but in fact they don't, and since this is not part of the core of the class, I prefer to eliminate sources of trouble for them (which eventually become sources of trouble for me) using this.

Writing the extension in Perl

There are many good things that can be said about Perl, for this or for anything else. But in this case there's a thing that makes it ideal for writing extensions: git includes a Perl module called Git, which is a Perl interface to all the Git commands. This is distributed with git, so if you've got git, you've got this library.

The whole extension is not hosted in this GitHub repo; this will contain the most up-to-date version as well as documentation and other stuff.

So here's the preamble to the extension:

use strict;
use warnings;
use lib qw( /Library/Developer/CommandLineTools/usr/share/git-core/perl
            /usr/share/perl5 );
use Git;

use v5.14;

my $HELP_FLAG = "-h";
my $USAGE_STRING = <<EOC;
Uso:
    git iv objetivo <número> -- crea una rama para ese objetivo
    git iv sube-objetivo     -- sube al repo remoto la rama

    git iv $HELP_FLAG -- imprime este mensaje
EOC

The main caveat about the extension is that some flags will be handled by git itself. There are probably quite a few of those, but one of them is --help. git xxx --help will try to look up a manual page for git xxx. This is why above a different help flag is defined. And also a usage string, which is helpful when you don't remember the exact shape of the subcommands. In this case, I use git iv as the extension name and as interface to the stuff that needs to be made; but there are subcommands that will do different things. These are implemented later:

my @subcommands = qw(objetivo sube-objetivo);
push( @subcommands, quotemeta $HELP_FLAG);

die( usage_string() ) unless @ARGV;
my $subcommand = shift;

die "No se reconoce el subcomando $subcommand" unless grep( /\Q$subcommand/, @subcommands );

my @args = @ARGV;

I prefer not to include any dependencies; there are powerful command line flag libraries out there, but in this case, a single script is best. So you handle whatever comes after iv uniformly, be it a subcommand or a flag. But the issue with the flag is that it includes a dash -, so we wrap it so that it can be used safely in regexes. Like the one, for instance, 4 lines below: in case the subcommand at the front of the command line is not part of the list, it will bail out showing the usage string.

Anything after the subcommand will be gobbled into @args.

if ( $subcommand eq $HELP_FLAG ) {
  say $USAGE_STRING;
} else {

  my $repo;

  eval {
    $repo = Git->repository;
  } or die "Aparentemente, no estás en un repositorio";

  if ( $subcommand eq "objetivo" ) {
    die $USAGE_STRING unless @args;
    $repo->command( "checkout", "-b", "Objetivo-" . $args[0]);
  }

  if ( $subcommand eq "sube-objetivo" ) {
    my $branch = $repo->command( "rev-parse", "--abbrev-ref", "HEAD" );
    chomp($branch);
    $repo->command ( "push", "-u", "origin", $branch );
  }
}

Now it's a matter of processing the subcommand. If it's the flag -h, print the usage string; if it's any of the other subcommands, we need to work with the git repository.

$repo = Git->repository; creates an object out of the Git library we mentioned before that we will use to issue the different plumbing or high level commands. One of the subcommands will do a checkout: $repo->command( "checkout", "-b", "Objetivo-" . $args[0]); will convert itself to the equivalent command. You can even work with plumbing commands such as rev-parse to check the branch you're in and create that branch remotely, ad the other command does.

Concluding

Perl saves you a whole lot of trouble when writing this kind of thing. Besides, the fact that it will be most probably be installed in any system you use to develop (Mac, Linux or WSL) will save you trouble asking for prerequisites for this script.

Creating new Perl composite actions from a repository template

Juan Julián Merelo Guervós — Wed, 22 May 2024 12:06:18 +0000

So you want to create a quick-and-dirty GitHub actions that does only one thing and does it well, or glues together several actions, or simply to show off a bit at your next work interview. Here's how you can do it.
Let me introduce you to composite GitHub actions, one of the three types that are there (the other are JavaScript GHAs or container-based GHAs) and maybe one of the most widely unknown. However, they have several things going for them. First, they have low latency: no need to download a container or to set up some JS environment). Second, they are relatively easy to set up: they can be self-contained, with everything needed running directly on the description of the GitHub action. Third, you can leverage all the tools installed on the runner like bash, compilers, build tools... or Perl, which can be that and much more.
Even being easy, it is even easier if you have a bit of boilerplate you can use directly or adapt to your own purposes. This is what has guided the creation of the template for a composite GitHub action based on Perl. It is quite minimalistic. But let me walk you through what it has got so that you can use it easier

First, this action.yml describes what it does and how it does:

name: 'Hello Perl'
description: 'Perl Github Action Template'
inputs:
  template-input:  # Change this
    description: 'What it is about'
    required: false # or not
    default: 'World'
runs:
  using: "composite"
  steps:
    - uses: actions/checkout@v4
    - run: print %ENV;
      shell: perl {0}
    - run: ${GITHUB_ACTION_PATH}/action.pl
      shell: bash

You will have to customize inputs as well as outputs here (and, of course, name and description), but the steps are already baked in. It even includes the correct path to the (downloaded) Github action: when you're acting on a repository, the place where a GHA is is contained in an environment variable, GITHUB_ACTION_PATH. You can access it that way.

In general, that script might need external libraries, even your own, which you might have moved out of the script for testing purposes (everything must be tested). That is why the action also contains App::FatPacker as a dependency; that's a bundler that will put the source (action.src.pl), your library (lib/Action.pm) and every other module you might have used into a single file, the action.pl referenced above.

A Makefile is also provided, so that, after installing fatpack, typing make will process the source and generate the script.

And that's essentially it. Use the template and create your new (composite) action in just a few minutes!

How to generate a great website and reference manual for your R package

Juan Julián Merelo Guervós — Wed, 10 Apr 2024 08:52:15 +0000

Generating a website for your R package is always a great idea. If the package is based on some paper, it will help it get noticed and eventually used. And once you have a website, it's just as well to include a reference manual for the package in it, that complements or is a bit more updated than the one published in CRAN. Or simply in another format.

That's what I tried to do for my package dupNodes. I found pkgdown, an one-stop solution for generating websites, which even generates the GitHub action for you.

I had a problem, though. I already had some semblance of website, using Jekyll to generate the pages from markdown, and just plain HTML elsewhere, for presentations and other stuff. And pkgdown generated an index.html, overwriting at least the main stuff, and not including some images I had in the README.md file.

So I decided to combine both in the same workflow. Here's what I had initially

name: Deploy Jekyll with GitHub Pages dependencies preinstalled

on:
  push:
    branches: ["master"]

  workflow_dispatch:

permissions:
  contents: read
  pages: write
  id-token: write

concurrency:
  group: "pages"
  cancel-in-progress: false

jobs:
  # Build job
  build:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Setup Pages
        uses: actions/configure-pages@v4
      - name: Build with Jekyll
        uses: actions/jekyll-build-pages@v1
        with:
          source: ./
          destination: ./_site

This is the default GitHub action generated by GitHub itself, pretty vanilla stuff. What it says is that it generates the HTML stuff to be deployed in a subdirectory called _site.

So far so good. The only thing you have to do is to tell pkgdown to generate it stuff in the same directory (or subdirectory thereof), right?

Wrong. jekyll-build-pages uses a docker container for generating the image, and that one does not follow the best practice of using a non-privileged user. Everything generated in _site has root privileges, so pkgdown failed with an EACCESS error.

So I had to add this step after that:

      - name: Change permissions built site
        run: sudo chown -R runner _site

Simple enough, but required some research to check who was the user running the GitHub action. After that, it was easy to incorporate the rest of the GitHub action, copypasta from the one generated by pkgdown itself:

      - uses: r-lib/actions/setup-pandoc@v2

      - uses: r-lib/actions/setup-r@v2
        with:
          use-public-rspm: true

      - uses: r-lib/actions/setup-r-dependencies@v2
        with:
          extra-packages: any::pkgdown, local::.
          needs: website

      - name: Build site
        run: pkgdown::build_site_github_pages(new_process = FALSE, install = FALSE, dest_dir="_site/docs")
        shell: Rscript {0}

The only change over the original is the dest_dir right above this: it's generated to _site/docs, within the _site directory that will be packaged for deployment.

Just one thing left. pkgdown does not seem to be aware of images linked from the README.md when it generates the cover for the site. So these had to be added too:

      - name: Link images
        run: cp -r _site/img _site/docs/

      - name: Upload artifact
        uses: actions/upload-pages-artifact@v3

Initial version used symbolic links, but upload-pages-artifact choked on it, so I had to physically copy the directory.

And that's it now. A website with reference and articles, updated automatically, without having to wait for CRAN updates!

Destructuring for type safety in JavaScript

Juan Julián Merelo Guervós — Tue, 25 Jul 2023 09:50:25 +0000

JavaScript allows you to use classes for modularization, but it's not easy to make a function just work with a single type. Let's see an example, taken from NodEO, an evolutionary algorithm library, now in a heavy refactorization phase.

export class StringChromosome extends Chromosome {
  constructor(aString, fitness) {
    super(fitness);
    this.stringChr = aString;
  }
  // more stuff
}

export class FloatChromosome extends Chromosome {
  constructor(aVector, fitness = 0) {
    super(fitness);
    this.floatVector = aVector;
  }
  // more stuff here
}

A chromosome has a common fitness, that is, how well it solves the problem, plus a data structure that represents the problem; EAs don't tell you which data structure you should use, so different types of chromosomes will have different data structures; we will use different names for those attributes, reflecting what they actually are.

Chromosomes undergo mutation; they essentially change randomly, looking for a better solution. A first version of a mutation operator would look like this:

// defined within FloatChromosome
static mutationRange = 0.2;
tatic mutate(chromosome) {
    const floatVector = chromosome.floatVector;
    const mutation_point = Math.floor(Math.random() * floatVector.length);
    let temp = [...floatVector];
    temp[mutation_point] =
      temp[mutation_point] -
      this.mutationRange / 2 +
      Math.random() * this.mutationRange;
    return temp;
  }

This works as expected, returning a new data structure that can be used to build a new chromosome (chromosomes are immutable, so it will need something else to compute the fitness before we build one).

However, chromosome might or might not be a FloatVector, so we would need to add some type checks.

if (chromosome.constructor.name !== "FloatChromosome") {
      throw new Error(
        `${chromosome.constructor.name} is not a FloatChromosome`
      );
    }

This will throw if we try to mutate a chromosome that has not been built as a FloatChromosome. So this code:

const sChrom = new StringChromosome("0001", 1);
console.log(FloatChromosome.mutate(sChrom));

Will throw:

file:///home/jmerelo/Code/js/dev.to-js-types/lib/chromosomes.js:30
      throw new Error(
            ^

Error: StringChromosome is not a FloatChromosome
    at FloatChromosome.mutate (file:///home/jmerelo/Code/js/dev.to-js-types/lib/chromosomes.js:30:13)
    at file:///home/jmerelo/Code/js/dev.to-js-types/script/mutate.js:7:29
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

However, these type checks are neither complete (what if it does not have a constructor?) nor precise (what if it does actually have a floatVector attribute, even if it's not been built with that class?).

const notReallyAChrom = { floatVector: [0, 0, 0, 1] };
console.log(FloatChromosome.mutate(notReallyAChrom));

This will still throw, although we would have been perfectly able to work with it:

file:///home/jmerelo/Code/js/dev.to-js-types/lib/chromosomes.js:30
      throw new Error(
            ^

Error: Object is not a FloatChromosome
    at FloatChromosome.mutate (file:///home/jmerelo/Code/js/dev.to-js-types/lib/chromosomes.js:30:13)
    at file:///home/jmerelo/Code/js/dev.to-js-types/script/mutate.js:7:29
    at ModuleJob.run (node:internal/modules/esm/module_job:194:25)

Destructuring FTW

We can easily fix this using destructuring for the function arguments. Let's define the function so that it extracts from the incoming data structure just what we need, and only what we need:

static mutate({ floatVector }) {
    if (floatVector === undefined) {
      throw new Error("Incorrect data structure: no floatVector attribute");
    }
    const mutation_point = Math.floor(Math.random() * floatVector.length);
    let temp = [...floatVector];
    temp[mutation_point] =
      temp[mutation_point] -
      this.mutationRange / 2 +
      Math.random() * this.mutationRange;
    return temp;
  }

This script:

const fChrom = new FloatChromosome([0, 0, 0, 0], 0);
console.log(FloatChromosome.mutate(fChrom));

const notReallyAChrom = { floatVector: [0, 0, 0, 1] };
console.log(FloatChromosome.mutate(notReallyAChrom));

const wrongChrom = new StringChromosome("010", 3);
console.log(FloatChromosome.mutate(wrongChrom));

Will work correctly until it finds the last chromosome. In that case, it will produce an error. By using destructuring we find (kind of) type safety, since we will reject all invalid data structures at the same time we accept data structures we can work with, whether they are declared as a class or not.

Coda

If you really want type safety, you should go for the real type-safe JavaScript, that is, TypeScript. If you need to stay with JS for some reason (and there are many good reasons to do so), you can achieve a certain kind of cleanliness using this programming pattern.

You can find the final version of the code used in this paper in this repo. Previous versions can be found in the commit history.

Follow that ship! With AISstreamer

Juan Julián Merelo Guervós — Wed, 10 May 2023 16:31:23 +0000

Imagine you're doing OSINT to track the shenanigans of some ship, or simply want to know where your family, who is cruising on the Mediterranean, is more or less precisely once their phones lack signal in the (more or less high) seas... Well, the Automatic data system AIS has you covered. It's mandatory that ships have automatic transmitter that broadcast data on where the ship is, as well as a certain amount of additional data, like destination.

This data is available in several per-pay and free APIs. A good one, which is for the time being experimental, is AISstream. This is a streaming API: you connect to a websocket, that sends you event; it makes sense, since continuous polling would probably bring down any server. Still, you have to set up some stuff, look for the correct library, and so on... AISstreamer (which tries, and fails, to create a portmanteau between steamer and streamer) has you covered.

Install it via

npm i aisstreamer

And you're good to go with this:

import { AIStrack } from "AISstreamer";
/** API_KEY needs to be defined as an environment variable */
const API_KEY = process.env.AISSTREAM_API_KEY;

/** The exact ship name needs to be used; this might include the company name */
const SHIP_NAME = process.env.SHIP_NAME.toUpperCase();

AIStrack(API_KEY, SHIP_NAME);

There's a single function in the API. It takes two mandatory arguments, the API key (which you will have assigned to the environment variable AISSTREAM_API_KEY) and the ship name (environment variable SHIP_NAME). AISStrack will print the whole message that includes this ship, via its default callback that simply does that, printing the variable; the other default is the bounding box where you want to look: by default it's the whole world, you can reduce it and pass it as the third argument.

Since the default callback function prints the latest AIS message the ship has emitted, it will go more or less like this:

 {
  Message: {
    ShipStaticData: {
      AisVersion: 2,
      CallSign: 'ZGIJ3',
      Destination: 'ANTIBES',
      Dimension: [Object],
      Dte: false,
      Eta: [Object],
      FixType: 3,
      ImoNumber: 9794549,
      MaximumStaticDraught: 3.3,
      MessageID: 5,
      Name: 'PAPA',
      RepeatIndicator: 0,
      Spare: false,
      Type: 37,
      UserID: 319156300,
      Valid: true
    }
  },
  MessageType: 'ShipStaticData',
  MetaData: {
    MMSI: 319156300,
    ShipName: 'PAPA',
    latitude: 43.587181666666666,
    longitude: 7.130946666666667,
    time_utc: '2023-05-10 11:46:52.509865357 +0000 UTC'
  }
}

Generally you will find two types of position messages: ShipStaticData, which, if I understand it correctly, is emitted when the ship is anchored, and PositionReport. AISstream does a good job of picking up earth stations, but apparently not satellites, so you will not always be able to track the position, only when it's close to one of them. At any rate, the most interesting piece of information is in the MetaData key, which is common to all messages: you have the latitude and longitude, which tell you where the ship is. From this static data you can also glean a couple of bits of data: Destination as well as Eta or estimated time of arrival (I guess, didn't check this much more).

The PositionReport do have a TrueHeading and NavigationalStatus, which you can also use to predict future position, I guess (I haven't really tried).

The possibilities are endless, from the simple, which is tracking and saving a ship's position, to the more advanced, like tracking a whole fleet.

And this is just the first version; future versions might include multi-ship tracking, automatic messages based on type of message, and a Lot of Cool Stuff (as well as your suggestions here or anywhere.

Testing your Raku module using Github Actions

Juan Julián Merelo Guervós — Fri, 14 Jan 2022 18:26:18 +0000

Using the Raku test GitHub Action (here in the marketplace), copy this into a test-raku.yaml file:

on: [ push, pull_request ]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Test via install
        uses: JJ/raku-test-action@main

That will install (and cache) dependencies before testing, and then do the test.

That's it ⃰.

⃰ I guess a bit of a making of is in order.

This is what is called a composite GitHub action. These types of actions enable the possibility of doing several steps, and we needed that, mainly to take care of the cache. How to do so was the subject of my previous post. While the cache is going to be generated outside the container, the rest of the action is going to run inside a container. Except for setting up the cache, the rest is run inside a container that runs this image and that has been designed specifically for GitHub actions (using what's mentioned above).

What happens if this one-size-fits-all does not fit you? Well, you can still use Github::Actions, that comes with its own container. But I guess that'll be the topic of a different post.

Designing containers for GitHub actions

Juan Julián Merelo Guervós — Thu, 30 Dec 2021 12:11:01 +0000

GitHub actions are workflows triggered by GitHub events. They are quite flexible, they run by default in "bare" operating system runners, but, additionally, they can be hosted in user-provided containers. The problem is, GitHub expect these containers to have certain services and structure, which is a problem if you use a standard or plain container.

Through the creation of this container, and multiple questions in the GitHub community, I've kinda discovered what are the things needed for containers to be used seamlessly in workflows. Here're the tips:

GHA user is UID=1001

While the first user you will create in any Linux system will have UID equal to 1000, GHAs are a bit special and they use UID 1001. So you'd better use that UID for your user, as is done in this Raku container

FROM alpine:latest as base

ARG RAKU_RELEASE=2021.12

ENV PKGS="git make gcc musl-dev perl linux-headers bash"

RUN apk update && apk upgrade \
    && apk add --no-cache $PKGS \
    && git clone --depth 1 --branch ${RAKU_RELEASE} https://github.com/MoarVM/MoarVM.git \
    && cd MoarVM \
    && perl Configure.pl --prefix /usr \
    && make --print-data-base \
    && make install\
    && cd .. \
    && git clone --depth 1 --branch ${RAKU_RELEASE} git://github.com/Raku/nqp.git \
    && cd nqp \
    && perl Configure.pl --backends=moar --prefix /usr \
    && make install \
    && cd .. \
    && git clone --depth 1 --branch ${RAKU_RELEASE} https://github.com/rakudo/rakudo.git \
    && cd rakudo \
    && perl Configure.pl --backends=moar --prefix /usr \
    && make install \
    && ls /usr/share/nqp/

FROM alpine:latest
ARG UID=1000

LABEL version="0.5.0" maintainer="JJMerelo@GMail.com" raku_release=${RAKU_RELEASE} raku_user_uid=${UID}

COPY --from=base /usr/lib/libmoar.so /usr/lib
COPY --from=base /usr/share/nqp/ /usr/share/nqp
COPY --from=base /usr/share/perl6/ /usr/share/perl6
COPY --from=base /usr/bin/moar /usr/bin/nqp /usr/bin/raku /usr/bin/perl6 /usr/bin/rakudo /usr/bin/


RUN mkdir /github \
    && addgroup -S raku  && adduser -S raku -G raku --uid ${UID}

USER raku
WORKDIR /home/raku

ENTRYPOINT ["raku"]

It's a bit longish, but the real deal is just above these lines. You can use that UID by default, but in my case I wanted two versions, just in case I needed more modifications later for GHS

Spoiler: I did

So I use an ARG that carries the UID, with 1000 by default, and there's a specific build-arg for GHAs. And that's that.

Is that all? Not really.

GHAs reset `HOME`

The container mounts a directory into /github/home and then calls it $HOME. Don't really know why, but it does. This can wreak a bit of havoc in languages for containers that expect home to be where it was originally used to install whatever. That can be easily fixed by just setting HOME manually at a job level, but it can also be fixed adjusting the search path for installed modules, which in the case of Raku means setting RAKULIB, as done here:

FROM ghcr.io/jj/raku-gha

ENV PKGS="git tar" PKGS_TMP="make gcc linux-headers musl-dev" WORKDIR="/home/raku"
LABEL version="1.0.3" maintainer="JJMerelo@GMail.com" rakuversion=$VER

USER root
RUN apk update && apk upgrade && apk add --no-cache $PKGS $PKGS_TMP
USER raku

# Environment
ENV PATH="${WORKDIR}/.raku/bin:${WORKDIR}/.raku/share/perl6/site/bin:${PATH}" \
    ENV="${WORKDIR}/.profile"\
    RAKULIB="inst#/home/raku/.raku"

# Basic setup, programs and init
WORKDIR $WORKDIR
RUN git clone --depth 1 https://github.com/ugexe/zef.git \
    && cd zef && raku -I. bin/zef install . \
    && zef install Linenoise \
    && cd .. && rm -rf zef

USER root
RUN apk del $PGKS_TMP
USER raku

ENTRYPOINT ["raku"]

This is already GHA-specific, as revealed by the base image. $RAKULIB is defined midway through the file, with the inst# prefix indicating it's already precompiled and ready to go. Raku will search there first, and then proceed to the supposed place, inst#/github/home/.raku. Where there will be noting in the raw container. Will it always be empty? Not by a long shot, because it will be used to install distributions by Raku, and thus it will be the place to cache. But something else is needed first.

Check also that we're including git in this image. Also useful for the checkout action (although this one has all kinds of defaults, and it's not actually essential).

You need to install GNU tar first

Alpine is an amazing little distro for containers, but part of its appeal comes from the fact that it puts a lot of external utilities into BusyBox, and that includes tar. However, in an apparently undocumented move, the caching GHA uses BSD or GNU tar for storing and restoring artifacts. That means it's needed in your container. Check above:

ENV PKGS="git tar" PKGS_TMP="make gcc linux-headers musl-dev" 
// and later
RUN apk update && apk upgrade && apk add --no-cache $PKGS

By installing tar, the image will be a bit heavier (in this case, taking 10 seconds to download and start), but not really a big deal.

And you're ready to go

See it in action here. Once the container is GHA-ready, you can run (at least these) Github actions in your workflow as easily as if you were running the default base runner.

How to totally over-engineer your CV, part two

Juan Julián Merelo Guervós — Sat, 20 Mar 2021 09:02:04 +0000

In Part I of this series we focused on the text part and how to generate it automatically using GitHub actions.
What would be a resume without a totally unnecessary programming timeline part, though? Something to show that you've been there, done that, gone through Pascal, you even used Windows in the 90s, and, well, a visual representation of your skills that gives an idea of how you can approach, and defeat, any JavaScript framework you're thrown at.
So let's start with that timeline in a CSV file, something like this:

Item, Group, Start Date, End Date
Basic, Languages, 1983-12-25, 1990-12-01
Pascal, Languages, 1988-09-01, 1993-01-01
C, Languages, 1989-01-01, 1993-12-01

... and so on. There's the item itself, the stuff you have experience with, then a group. I have Languages, Environments like Linux or cloud, and then Tools like Docker; suit yourself here.

You might want "Sewing techniques" like "needlepoint" or "macramé" or "Zombie vanquishing" "with baseball bat" "sawed-off shotgun", whatever. Hey, any job, up to and including zombie exterminator, is well served with an over-engineered CV.

Start and end date are self-describing, only I am going to be using December this year, for purely aesthetic reasons, to indicate the stuff I'm still using now.

I will render this using R; as usual, there are good R visualization libraries for mostly anything. I will use vistime, an R module for visualizing timelines. But with a twist:

library("vistime")
library("ggplot2")

data <- read.csv("data/programming.csv")

g <- gg_vistime(data, col.event="Item", col.start="Start.Date", col.end="End.Date", col.group="Group") + theme(axis.text.x = element_text(angle=90, color='blue4',size=14) )+coord_flip()

g.d <- ggplot_build(g)

g.d$data[[4]]$angle <- 90

rebuilt <- ggplot_gtable(g.d)

png(filename="img/timeline.png", width=240, height=960)
plot(rebuilt)
dev.off()

The twist is that I wanted a vertical timeline, instead of an horizontal one; basically to fill one of the margins of one of the pages which was sitting empty, instead of occupying a big part of a page with a regular, horizontal, timeline. That's done in the second statement: a ggplot2 data structure is generated, which is basically a timeline with the coord_flip statement that flips coordinates to make it vertical. However, the problem was that bar labels were not flipped. Besides, since some items started at pretty much the same time, they overlapped. The result was not nice, and had the right-level-of-engineering. So we had to overengineer it with the next 6 sentences.

I knew that I had everything that's in the chart available in the g data structure. Any other language, it would have been easy to dis-assemble that thing into its composed objects, and deal with the part that actually does the labeling. Not so easy with R. In order to have access to the data structure, you need to issue the ggplot_build order.

Deconstructing through a command called build, that's rich...

That creates a table with a series of data frames, and you can work with that. The fourth element

and yes, R starts its arrays with 1, just like Pascal. See? Learning Pascal was not so useless, after all (in fact, it was pretty useful and the best thing you could do in the 80s).

contains a dataframe with all the bars. Flipping them is as easy as changing the angle to the right one, which happens to be the right angle too:

g.d$data[[4]]$angle <- 90

After that, you still have a dis-assembled chart, which you need to assemble back, and then actually save to a file to have it ready.

Running that is simply a matter of issuing a command from the command line. Creating a workflow that generates it is another matter. This might be the first version you could think about

name: "Programming timeline"
on:
  push:
    paths:
      - 'data/*.csv'

jobs:
  creates-timeline:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Instala R
        uses: r-lib/actions/setup-r@v1
        with:
          r-version: '3.5.3'
      - name: Install packages
        run: sudo apt-get install libcurl4-openssl-dev
      - name: Install dependencies
        run: |
          install.packages(c("ggplot2", "curl", "httr", "plotly", "vistime"))
        shell: Rscript {0}
      - name: Ejecuta el script
        run: Rscript .github/workflows/timeline.R

Off the bat, pretty straightforward. Set up R using one of its version managers, install the downstream dependency packages you need, then install the actual packages that are going to be using (including some dependencies that should probably not be there but are anyway, to check that they are installed in the right order). And then run the script.

That takes 7 minutes. That's a lot for a timeline. Installing R packages includes compiling some C, and sometimes Fortran, source, plus all the rest. 7 minutes. We need to cut that down, if only to not include in the timeline a substantial part that says "Generating this timeline".
Again, using a cache was discarded off the bat. Too much stuff in too many different places. But this time using a Docker image did make sense. That image will include pretty much the same, only it will come in a single, convenient, package:

FROM r-base

LABEL version="0.0.1" maintainer="JJMerelo@GMail.com"

RUN apt-get update \
    && apt-get install -y libcurl4-openssl-dev r-cran-ggplot2 libssl-dev r-cran-httr git\
    && R -e "install.packages(c( 'plotly', 'vistime'))"

WORKDIR /home/docker

ENTRYPOINT ["Rscript"]

Not a lot of overengineering here; just a straightforward installation of needed packages, and an entry point that enables us to use it as a R script runner.

Did I say not a lot of overengineering? Well, just the right amount. This is public in Docker Hub, which will try and compile a new image every single time you do a push. In most cases, there will be nothing new in the Docker image, it will just be some data added to the CV. And yes, I could have a path-filtered Github Action that uploaded it either to Docker Hub or to GitHub registry? But why? It's quite simple to do just the compilations you need, using Docker Hub hooks written in Perl, like this one:

use Git;

my $repo = Git->repository (Directory => '.');
my @modified_files = $repo->command('diff', "--name-only", "HEAD", "HEAD^");
die("No Dockerfile modified in the last commit\n")  unless grep( /Dockerfile/, @modified_files);

This one is actually pretty standard, and can be used in any repo as long as your Dockerfile is in the root dir. It checks if Dockerfile is in the last commit, and dies if it does not, signalling the Docker Hub pipeline that it should not continue.

This appears as a failure in the pipeline; there's no way to make the pipeline just stop instead of failing.

Again, this is possible because Perl is also installed in the Docker Hub runner. It's everywhere.

We can then use this Docker image-that's-only-regenerated-when-it-needs-to to in a renewed GitHub action:

name: "Programming timeline"
on:
  push:
    paths:
      - 'data/*.csv'

jobs:
  creates-timeline:
    runs-on: ubuntu-latest
    container: jjmerelo/cv
    steps:
      - name: Checkout
        uses: actions/checkout@v2
      - name: Ejecuta el script
        run: Rscript .github/workflows/timeline.R
      - name: Checks in results
        shell: bash {0}
        run: |
          if [[ $(git status -s) ]]; then
              git config --global user.email "jjmerelo@gmail.com"
              git config --global user.name "CVDataBot"
              git commit -am "Update timeline chart"
              git push
          else
              echo "🟏 No Changes"
          fi

The bulk of it is actually checking if there's been some change in the image (even if the data changes, maybe the image does not, for instance, if it's only a change in whitespace). But there's a small piece of overengineering there too, we'll get to that later.

As shown in the container key, this GitHub action is run inside the container, so what's going to be available there is only what you put into the container. This is preferred to running it inside any other container and then docker pulling the image; it saves a bit of time. But then it runs the source checkout in the container too. This is a clever piece of software: it will run no matter what; it will download your source using its own devices... But we need to checkout the actual git repository, because we will need to push to it afterwards. This is why the container uses this:

apt-get install -y libcurl4-openssl-dev r-cran-ggplot2 libssl-dev r-cran-httr git

Installing git where it, in principle, would not have been needed. If present, actions/checkout uses git, and then we have our repo available for committing and pushing and whatever!

And it's also got another goodie: bash. By default, the shell used in a GitHub step is sh. You can change that, however (yes, including Perl, and yes, I added that piece to the documentation through a PR), and since doing the expression and checking with shell was a bit of a nuisance (and no, we were not going to install Perl in that container (although, come to think of it, it might be there already...))

... Anyway, an alternative way of just showing off that you know not only bash, but the difference between the syntax of bash and plain sh.

This new version takes less than a minute, with the bulk of it going to downloading and setting up the container

With a bit of more over-engineering, we could create a slimmed-down R container... Maybe in the future.

With that, your just-in-time generated CV is ready for downloading

Was it worth the while?

Well, creating a CV is positively boring. Using what you know to make it prettier or automatize its generation, and make it also a bit more entertaining, is definitely worth the while. It also shows craftsmanship, and a passion for optimization.

Also a bit of a tendency to overengineer stuff that, again, could be created using a wordprocessor. But that's not always bad, right?

How to totally over-engineer your CV in a few easy steps

Juan Julián Merelo Guervós — Fri, 19 Mar 2021 09:32:20 +0000

Once you need (or don't really need) to create a CV, do it real good, making it show off a few skills. Instead of a static file, make it a dynamic showcase of a few things that you have learned in your career and that might be interesting for the job you've applied to.

Or simply show that you like overengineering something that's mostly done using a word processor. Anyway.

Anyway, here are the yaks I shaved when creating this CV, which, just in case you wondered, is my (more or less real) brief resume, practically up to date. It's got a free license, so feel free to fork the repo (and also add a few pieces more of over-engineered stuff).

Use (or adapt) your own CV template

There are many LaTeX templates out there you can use. What they essentially do is pump up headers, and of course give the finished CV a specific appearance. Programming LaTeX is exactly in 0 job postings, but picking up something in a totally unknown language and adapting it to your needs is indeed a nice skill. Anyway, here's the result where basically a few fonts and colors were changed, so that they're adapted to modern Ubuntu packages, mostly.

Check out git blame just in case you want to see exactly what was changed, and maybe use the original.

Use XeLaTeX and biber, not LaTeX

Well, that came with the original template, so, well, it wasn't really much of a choice. What's nice about using biber is that it's a Perl script, and you need to get the correct CPAN package and Perl version to get it working.

Generate the PDF automatically

It would be a waste of computing time to type make pdf or anything like that to actually generate it, right? Actually, it's more complicated than that. Best practices advise you to not include generated stuff in your repo (mostly), and this PDF would clearly be a product of your application (the CV generator). So we could as well create real releases, that have the CV attached to them. We do so with this Github action:

name: "Generate CV"

on:
  push:
    tags:
      - '*'

jobs:
  generate-cv:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v2
        with:
          fetch-depth: 0
      - name: Install tools
        run: .github/workflows/latex-install.sh
      - name: Generate pdf
        run: ./generate_cv.sh
      - name: Go to main branch and generate thumbnail
        run: git status && git checkout master && thumbpdf --makepng --verbose --noclean cv.pdf && mv thb1.png cv.png
      - name: Publishes CV
        uses: ncipollo/release-action@v1
        with:
          artifacts: "cv.pdf"
          body: Fresh CV generated
          token: ${{ secrets.GITHUB_TOKEN }}

It works only when you tag a repository (and remember to git push && git push --tags so that the actual commit that's tagged is used). It needs to check out the whole repo, and not only download the tagged code, for reasons that will be clear a bit later. But we need to install the tools:

sudo apt-get install texlive-xetex biber fonts-goudybookletter texlive-fonts-extra  texlive-science-doc texlive-science fonts-adf-accanthis fonts-smc-gayathri ghostscript

Essentially it's installing a bunch of packages, including the fonts that were declared in the LaTeX class file at the beginning. And ghostscript, whose need we'll see a bit later. This takes a good while, around 4 minutes. So it's ripe for optimization, one way or the other. We could dockerize it, but we'll save dockerization for later. We could also use a cache. And that we tried.

We really over-engineered this one...

This version uses a caching github action step. Wasn't really worth the while. It took like a minute to check for cache hits, and around two minutes to actually download the cache in case of cache misses, not to mention around half an hour to actually set up the cache in the first place. All that to shave a bare minute... Not really worth the while. That way we can change more things in the Latex setup if we want.

Actually generating the PDF is relatively straightforward:

xelatex cv.tex
biber cv
xelatex cv.tex

Except that, remember, biber uses Perl. So how come this work? Well, perl is installed in all Linux runners in GitHub actions. We'll see more of that later on... So this works without a glitch.

After that, we can create the release. Jump over the next step (which is there for historical reasons) and go to the next one, which simply Publishes CV using another external github action. Very simply, it picks the generated artifact, cv.pdf, creates the release and attaches it to it.

But we need to generate a thumbnail, because of course, we need a thumbnail to link to the actual CV. I said this was going to be overengineered, right? This will be used in the index page to point to the actual page that will be used to download the CV. This does the trick:

git status 
  && git checkout master 
  && thumbpdf --makepng --verbose --noclean cv.pdf 
  && mv thb1.png cv.png

Actually, the last two lines do; the first are there to go from the tag to the main branch. After exploring lots of different utilities, like convert, part of ImageMagick, which is actually broken, thumbpdf, which is installed alongside LaTeX but actually uses ghostscript underneath, works nicely, generating a PNG for each page. The last command calls it the way we want.

thumbpdf is also written in Perl. Now you start to understand why Perl is installed everywhere (and why it's always a good thimg (TM) to know how to use it. As a matter of fact, I had to check out the source to find out what was missing after having it fail with an obscure error message.

This is actually a generated file, and we've included it in the repo. Well, why not. It's a small file, images (like this, which is actually an icon) will be needed for the page (and you can see it above). But just having it in this local copy of the repo is not enough. We will need to push it to the repo.

But before, we'll also create other things we need to push to the repo. The CV is going to be uploaded to a different URL every time a release is created. Although the template for such URL is straightforward, and can be found by simply navigating the site, we need something much simpler: generate a page with a link that you can simply click to download de CV. Perl again to the rescue, in these two final steps of the action:

      - name: Generate URL
        run: .github/workflows/generate-download-url.pl > DOWNLOAD.md
      - name: Pushes thumbnail and URL file.
        env:
          USER_NAME: JJCVBot
          EMAIL: jjmerelo@gmail.com
          MY_COMMIT_MSG: Updates download doc
        run: curl https://raw.githubusercontent.com/JJ/perl-GitHub-Actions/main/bin/commit-push.pl | perl

The first script effectively generates the markdown file that will be checked in, and that will be rendered to web and used effectively as a signpost to download the CV. Here's that iw does:

use strict;
use warnings;
use constant URL_FILE => "download_cv_url.txt";

use Git;

use v5.14; # For say
my $repo = Git->repository (Directory => '.');
my @tags = $repo->command("tag");

my $tag = pop @tags;

my $download_url = "https://github.com/JJ/cv/releases/download/$tag/cv.pdf";

open my $url_file, ">", URL_FILE;
print $url_file $download_url;
close $url_file;

say <<EOC;
# Download CV
[Download latest version of CV in PDF]($download_url)
EOC

First interesting thing about this is that it's run directly on the Github Actions runner, no external dependencies. It uses Git, a Perl module installed alongside git proper. That's used to get the tags in the repo (remember, we checked out the whole repo) and use the last one to generate the URL that's printed to a file, and also to the DOWNLOAD.md markdown file.

The tag that triggered the action is actually available as an environment variable, so we could have used that. However, we didn't know in principle if we were going to use that kind of trigger. This script will work even if we trigger the action in some other way.

Finally, we're checking this in to the repository; there are three files: the download URL (which we could, in principle, use like this: wget $(wget raw_download_url)), the thumbnail and DOWNLOAD.md. We will use an external Perl script to do so.

This script is part of the GitHub::Actions perl module, which contains a few utility functions (and scripts) to use within your, you guessed it, Github Actions. Since GHAs use environment variables to communicate, it takes the username and email (used to set the git configuration locally for the commit; you should use your own email if you want the commit to be identified as yours in GitHub) and the commit message. Here's the (rater uncomplicated) script:

use strict;
use warnings;
use v5.14;

use Git;

my $repo = Git->repository (Directory => '.');

my $user_name = $ENV{'USER_NAME'} // $ENV{'GITHUB_ACTOR'};
my $email = $ENV{'EMAIL'};
my $commit_msg = $ENV{'MY_COMMIT_MSG'} // "No message";
if ( $repo->command( 'status', '-s' ) ) {
  $repo->command('config', '--global', 'user.email', $email);
  $repo->command('config', '--global', 'user.name', $user_name );
  $repo->command('commit', '-am', $commit_msg );
  $repo->command('push');
}

Again, it's perusing the Git module which we know is available, there are a whole lot of Perl modules availables there, doing a whole lot of things. If you can use them, you will save some time because you will not need to pack them somehow. Anyway, it configures the email and name, commits and push. Couldn't be more straightforward. You could actually do that easily with a shell script, pretty much this way:

if [[ $(git status -s) ]]; then
              git config --global user.email "jjmerelo@gmail.com"
              git config --global user.name "ÁgilDataBot"
              .github/workflows/commit-msg.pl | git commit -aF -
              git push
          else
              echo "🟏 No changes"
          fi

Essentially, you will need to avoid making a commit and push if there are no changes, because the step would error (it could happen if you have pushed to an existing tag, for instance, and the cv has not changed).

That's it for the resumé

However, there's an important part of the resumé which is your programming timeline. That will need a whole 'nother article, I guess. You can check it out in part II of this series

ALL environment variables available in the Docker Hub build environment

Juan Julián Merelo Guervós — Mon, 28 Dec 2020 10:05:21 +0000

The Docker Hub documentation lists all these environment variables, which are available in your scripts in that environment:

    SOURCE_BRANCH: the name of the branch or the tag that is currently being tested.
    SOURCE_COMMIT: the SHA1 hash of the commit being tested.
    COMMIT_MSG: the message from the commit being tested and built.
    DOCKER_REPO: the name of the Docker repository being built.
    DOCKERFILE_PATH: the dockerfile currently being built.
    DOCKER_TAG: the Docker repository tag being built.
    IMAGE_NAME: the name and tag of the Docker repository being built. (This variable is a combination of DOCKER_REPO:DOCKER_TAG.)

However, there are quite a few more, which might be useful in a certain environment; these were discovered by just printing the environment variables via a Perl script (which, BTW, you can use in Docker Hub for your own purposes).

SOURCE_TYPE: this one seems to contain git. It might be different if it's pushed directly, or via the API.
PATH: is the regular system PATH. It contains /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin.
MAX_LOG_SIZE: it contains 67108864. Not clear if it can be changed.
BUILD_CODE: no idea what this is. It contains a code.
PUSH: it contains true if it's been created via PUSH. Probably other values if it's been automatically launched.
GIT_TAG and GIT_SHA1: no idea what the first one is, the second is the commit SHA1.
HOSTNAME: the host it's using to build.
DOCKER_HOST: the socket the client is using to connect, same as in the local docker installation.
There's something called SIGNED_URLS, which holds Amazon urls. No idea what this is, or how it can be useful.
DOCKER_TAG and CACHE_TAG, the tag that's being built, like latest.
PYTHONUNBUFFERED is equal to 1.
SOURCE_REPOSITORY_URL the repository it's being built from.
LOG_BUILD_STEPS is set to FALSE
DOCKER_CFG contains information on the user, including user tokens.

Out of all these, you can probably use PUSH or even GIT_SHA1 to introspect what's being built, and maybe take some actions. For instance, along with information on the last build you could create a script that built only those Dockerfiles that have changed.

DEV Community: Juan Julián Merelo Guervós

You can also have (some) type safety in JavaScript. Here's how.

Destructuring for type safety in JavaScript

Juan Julián Merelo Guervós ・ Jul 25 '23

Writing extensions to git opens up lots of possibilities of enhancing teamwork through a commonly used tool. And #perl will make it easier.

Writing git extensions in Perl

Juan Julián Merelo Guervós ・ Feb 10

Writing git extensions in Perl

Introduction

Writing the extension in Perl

Concluding

Creating new Perl composite actions from a repository template

How to generate a great website and reference manual for your R package

Destructuring for type safety in JavaScript

Destructuring FTW

Coda

Follow that ship! With AISstreamer

Testing your Raku module using Github Actions

Designing containers for GitHub actions

GHA user is UID=1001

GHAs reset HOME

You need to install GNU tar first

And you're ready to go

How to totally over-engineer your CV, part two

Was it worth the while?

How to totally over-engineer your CV in a few easy steps

Use (or adapt) your own CV template

Use XeLaTeX and biber, not LaTeX

Generate the PDF automatically

That's it for the resumé

ALL environment variables available in the Docker Hub build environment

GHAs reset `HOME`