DEV Community: Andrea Giacobino

geo2tz - 4 years later

Andrea Giacobino — Sun, 30 Jun 2024 23:12:00 +0000

tl;dr after 4y, the projects have been substantially updated and it is now well-tested and mature.

In July 2020, I wrote on this platform about geo2tz, a rest API to retrieve the timezone from latitude and longitude coordinates.

I have sporadically updated the project from time to time, and now, four years later, something has happened that moved me to give it some love and make sure it is up to date, this led to a complete rewrite of the engine that powers it, and this post is about what are the reasons and what are the results of this rewrite.

When I published the project in 2020, I was working on another project, and I needed something like geo2tz, but I could not find anything that fit my requirements, so I decided to create it by putting together a web framework, the timezone data and a library that was providing the logic to process and query the timezones GeoJson, and that was it.

Fast forward to the beginning of 2023, I get an issue open of a person complaining about a set of coordinates missing, but it looked like it was a dataset issue, so there was nothing much to do. But at the beginning of 2024, someone pointed out that the service was not working properly for other coordinates.

Clearly, something was afoul, and since there were people who took the time to comment, I took it more seriously to check what was going on. What I found out is that the issues were coming from the library that I was using to manage the timezone data, the library stopped being updated and was actually returning incorrect results.

I took my sweet time to do it, not gonna lie, but eventually, I rewrote the GeoJSON parser, re-engineered the index and algorithm to match the timezone from coordinates, and added a lot of tests to make sure that geo2tz behaves correctly and here we have, a new shiny version (2.4.0) that is ready to put to use!

Rust + Wasm + Tailwind + Svelte

Andrea Giacobino — Sun, 03 Jan 2021 20:49:05 +0000

For this winter lockdown season I took some time to learn Rust and to pack it up with WASM, Tailwind and Svelte. The big absent tech here is TypeScript, but there is already a lot to unpack.

This post is about the experience of building the CostOf.Life project from scratch using the tech stack in the title (the sources for everything is provided in the project website).

CostOf.Life is a command line client and a web interface to model your personal finance over time by giving it a daily cost perspective.

Number #1: Rust

I have been exposed to rust for a couple of years by now, working alongside John and making a few (unsuccessful) tries to start a project myself.

Finally I made the step to read the rust book and boy was it worth it. I mean, Rust is the Dark Soul of modern programming languages, not because it is intrinsically difficult, but because how unforgiving the compiler is.

Turns out that I have now joined the chorus of the ones singing Rust praises: once you get the theoretical basis it is a joy to work with it and you start to really appreciate the compiler (and especially its suggestions); plus you get a memory safe language with no garbage collection¹!

Number #2: WebAssembly (Wasm)

This was a big one also driving the choice of using Rust in the first place, Rust is one of the language that has embraced WebAssembly the most (just search Wasm in GitHub) plus the tools to work with Rust+Wasm are just amazing.

Parity has done a lot of work with Rust+Wasm, Cosmos smart contracts are are written in Rust+Wasm, and a quick search for npmjs shows that it's very popular with crypto stuff commonly used in blockchain tech (fyi I work with Ape Unit a blockchain consulting agency in Berlin).

So what about it? The current most popular usage of Wasm (end of 2020) seems to be for malicious purposes. As I see it, for 99% of use cases for common web purposes Wasm is just not necessary, plus it's not so easily integrated (see point #4) so for the moment I am skeptic about adoption beside the crypto nice.

Number #3: Tailwind

As I mentioned there is a web interface for the subject project. I am definitely not a visual person, I cannot put together a website to save my life!

But then Enrico (that does that for a living), recommended to drop any fancy idea and just stick with tailwindcss and I have never looked back (it is that great).

Number #4: Svelte

If you are an adult in 2020 chances are that you know about Game of Thrones and web frameworks are like characters in the serie, there are many but few survive.

So we still have Angular for some reason I don't know, then React, the current king (Joffrey I guess ¯_(ツ)_/¯) and what you must choose when outsourcing, Vue the underdog and Svelte the snobby one.

I end up choosing Svelte: it is my first experience with any web frameworks and I must say that I have enjoyed very much working with Svelte + Tailwind + Rollup, but again, I am very novice in the field so not very relevant.

Putting all together

Building rust command line client, the rust library and building + testing + publishing the wasm npm package was a breeze. Fantastic tooling, super easy and straightforward.

Building the web app with Svelte + Tailwindcss also quite easy, the project is not very simple but anyway a nice de experience.

And now the ugly: I could not manage to actually use the published wasm package with Svelte.

There are some issues with rollup/webpack and wasm that I could not overcome.

But worthwhile are:

Rust+Wasm lib: https://github.com/noandrea/costoflife-rs
Wasm testing: 1, 2
Rollup+Tailwind: https://github.com/noandrea/costoflife.ui/tree/master

And of course, the CostOf.Life project.

I am sure in the following months the issues that prevent this experiment to be successful will be resolved, I would love to contribute to that but my knowledge in the field is not enough, but if you can and have the will, I would be thrilled to know your opinion on the matter!

I come from a Java background and I work mostly in Golang ↩

Get the upstream distro name in Rust

Andrea Giacobino — Wed, 26 Aug 2020 14:58:08 +0000

The other day we were chatting with a colleague of mine about retrieving the OS name of the distribution. I don't recall exactly the details, but it was something about always getting the actual distribution name and not the upstream one, and he could not sort that out (I have my suspicions that he didn't try very hard).

Fast forward 2 weeks, I'm sipping coffee and installing updates on my laptop and suddenly this appears on the screen:

Oh, that's interesting 🤔, let's google "rust distribution id_like"? LMGTFY

BOOM! first result: https://docs.rs/os-release/0.1.0/os_release/struct.OsRelease.html

Looks good, let's give it a try:

> cargo new guess_os
> cd guess_os 
> cargo add os-release
> vim src/main.go

very well now let's add the code

extern crate os_release;

use os_release::OsRelease;
use std::io;

pub fn main() -> io::Result<()> {
    let release = OsRelease::new()?;
    println!("You say '{}', I say '{}'", release.name, release.id_like);
    Ok(())
}

And now the moment of truth 🥁

🏆

Go vs Py

Andrea Giacobino — Fri, 07 Aug 2020 10:52:22 +0000

TL;DR Comparing Go and Python for code execution time, working with Docker images and native packages distribution.

Recently I was working on a programming challenge and I thought it would be interesting to have an implementation both in Go and Python.

The challenge consists of processing a dataset (specifically a csv of ~100mb), make some aggregations and output a json file that will be used as a data-source to serve a bunch of REST API endpoints.

What are we comparing?

But enough with the chit-chat, lets dig in about the comparison.

Execution time: time difference in speed between the two implementations to execute the data aggregation algorithm.

Docker image: complexity in producing a docker image and size of the generated images.

Native distribution: ease of use to install and run the app in a native environment (aka on your laptop).

A bit of context

Both app are available on github, here the Go version and here the Python one.

Specifically we'll be comparing PLZ Go v0.2.1 with PLZ Py v0.1.3

The Go and Python version are respectively go-1.13.8 and python-3.8.5.

Both implementation are quite small:

PLZ Go: 250loc
PLZ Py: 191loc (~23% smaller)

Metrics generated with tokei using:

for Go tokei -f -t=Go -e="*_test.go

for Py tokei -f -t=Python -e="tests"

Criteria #1: Execution speed

Both implementations use the same identical sequential algorithm to process the data and both input and output are the same.

Average speed¹ to process ~100mb csv input to a 32k json ouput is:

PZL Go: ~0.6s (~2.2x faster)
PLZ Py: ~1.3s

The algorithm implementation is in the massage function: here in v0.1.3 for Python and here in v0.2.1 for Go

If you dig into the code and you spot some evident problem affecting the performance please let me know in the comments.

Not much to say here, Go is twice faster than Python in this scenario.

Best: Go

Criteria #2: Docker image

Since the application provides a REST API to serve the results of the aggregation, it make sense to ship the app as a Docker image.

The approach to build the docker image is to use a multi-stage build: first build the app and aggregate the data and then assemble a "production" image.

For this criteria Go is hands down the best, you can instruct the compiler to build a self contained binary to be package in a scratch image for a resulting image size of little more of 9mb (3.5mb compressed).

On the other, with Python, the final image (based on 3.8-slim-buster) is a staggering 175mb (55mb compressed). This is due to the many layers that compose the 3.8-slim-buster base image, therefore there is room for improvement by building a custom image, but that will likely require a significant effort for build and maintenance.

PZL Go: ~9.3mb - (18x smaller)
PLZ Py: ~175mb

Best: Go

Criteria #3: Native distribution

By native distribution I mean to install the app on a server or on your laptop without having to tinker too much with software requirements. I strongly believe that this is an important metric, since having complex installation procedures involves a lot of cognitive effort that is ultimately wasted.

In this area both Go and Python are more or less equivalent on the surface, with Go you can install the package running go get github.com/noandrea/plz and with Python using pip install plzpy.

A caveat is that Python is far more popular and get shipped by default with many OS, while for Go you will likely have to download and install the Go toolkit.

Another notable difference is that with pip install you are installing a "binary" distribution while with go get you are fetching the source code and compiling it locally.

Worth mentioning that the maintainer(s) of the Go package may have shipped binaries for the OS specific architectures and package managers (brew, dnf, apt, ...), but that requires additional work and maintenance.

Slightly better: Python

Conclusions

I hope you enjoyed this comparison as much as I did in making it, and about which one is better, Go or Python, the answer is ....🥁 ... both and none 😉!

👋

consistent over multiple executions ↩

Terminal split-screen recording with asciinema + screen

Andrea Giacobino — Sun, 19 Jul 2020 19:11:38 +0000

While showcasing geo2tz I wanted to record both client and server on the same asciinema video.

Luckily you can do just that using screen.

I'm assuming at this point that you have installed both asciinema and screen and that you are on linux.

Before you start, add this line layout save default in your ~/.screenrc

Next setup your screen session so when you are about to record the screen will be already set:

screen -R devto

Once there execute this sequence of keystrokes (case sensitive):

ctrl + a + S // split horizontally
ctrl + a + Tab // move to the bottom window
ctrl + a + c // open a new shell
ctrl + a + Tab // move back to the top window
ctrl + a + d // detach (but not close) the session

pro tip, for maximum effect you can set the windows name with ctrl + a + A

Now that the stage is ready you can start recording:

asciinema rec -c "screen -R devto"

Here is how it looks like:

if you don't see the embed, follow this link to the video.

Do you know another way of doing this, maybe with tmux?

Self-hosted Geo to Timezone service

Andrea Giacobino — Wed, 08 Jul 2020 20:30:36 +0000

For the VALIS project I needed a geo to timezone service, specifically to pick up a reasonable time to send chat messages.

There are a few hosted services that can do that for you, but I believe that privacy is a feature, and I am not comfortable with sharing location data with 3rd party services.

A geo to timezone service is a simple service to implement, especially since the heavy stuff 💪 has been already implemented by better human beings 🙏 (the link above also lists the libraries).

And so here it is: geo2tz

Here is how to run and how to interact with it :

if you don't see the embedded video, follow this link

ps. docker images are hosted on gcr

Build a demo dataset for your product

Andrea Giacobino — Wed, 17 Jun 2020 10:42:14 +0000

I am working on a new product (btw it's name is VALIS) and we needed a fake dataset to showcase the product and it's functionalities, here some suggestion if you have to do the same¹

Start from a real world scenario that you know and you are familiar with

Faking stuff is hard, better to stick with what you know and experience directly: take the data from your personal or work life and use to create the canvas for your demo dataset, next you'll replace names and other details.

Make it familiar for your audience

Who is your audience? Are they young? Is gender relevant? You probably already have a good idea about that if your are building a product, keep this in mind when choosing the actors in your dataset.

Keep it as straightforward as possible

The goal of your demo dataset is to showcase your product, not your demo dataset, try to make evident what is what: for example, use well known identifiers for companies (LLC, Inc, ...), locations (City, Lake, ... ), projects, etc.

About using existing brands

Trademarks are a complicated issue, it may be fine to use existing brands or it may not, treat with care. When not sure, don't: better to avoid to recreate your demo and material because of trademark infringement.

The holy grail: tailored demo data

Can you think of a way of generating tailored demo data? maybe processing publicly available info from Wikipedia, GitHub or Crunchbase? Then go for it, showcasing a product using your client data will be way more effective than using unknown or fake data.

What about well known dataset?

It depends, well known for who? If your are not 100% sure that your audience is familiar with a topic is safer to assume that they are not.

And since your are here, check out VALIS!

it doesn't apply for every type of project 😉 ↩

From HackMD to PDF

Andrea Giacobino — Sat, 09 May 2020 08:42:49 +0000

Full disclosure, I'm a big fan of Markdown, and also a big fan of HackMD (and it's community fork CodiMD).

Now imagine you have to write a technical assessment or specifications and you started writing it down on hackmd/codimd because, you know, reasons 😤. But then, for the sake of the argument, let's say that you have to produce a deliverable artifact that can be shipped to someone, like a customer; then a link to a online document might just not cut it 🤔.

So what to do? Alright, easy solution, fire up typora (or similar), paste the markdown, File > Export > PDF. The day is saved.

You go back skimming through the news and you find this: Writing a Book with Pandoc, Make, and Vim

Wait a second... what if... 🤯 vim Makefile

AU_HACKMD_CODE  = bfScafhcTdGzzhJ4B3evlQ
AU_FONT         = NeufileGrotesk
AU_FONT_DIR     = $(HOME)/.fonts/NeufileGrotesk
AU_OUTFILE      = $(HOME)/$(AU_HACKMD_CODE).MD.$(shell date +%F).pdf
AU_FORMAT       = article

hackmd2pdf:
    @echo format doc
    curl -s -L "https://hackmd.io/$(AU_HACKMD_CODE)/download" -o /tmp/$(AU_HACKMD_CODE).md
    pandoc /tmp/$(AU_HACKMD_CODE).md -o $(AU_OUTFILE) \
  --table-of-contents \
  --pdf-engine=xelatex \
  --highlight-style=monochrome \
  --number-sections \
  -H $(HOME)/.config/apeunit/header-$(AU_FORMAT).tex \
  -V 'mainfont:$(AU_FONT)' \
  -V 'mainfontoptions:Extension=.otf, UprightFont=*-MediumExtended, BoldFont=*-BoldExtended, ItalicFont=*-MediumExtendedItalic, BoldItalicFont=*-BoldExtendedItalic, Path=$(AU_FONT_DIR)/' \
  -V 'fontsize: 12pt' \
  -V 'papersize: A4' \
  -V 'urlcolor: blue' \
  -V 'date: \today{}' \
  -V 'documentclass: $(AU_FORMAT)' \
  -V 'geometry:margin=3.8cm'
    xdg-open $(AU_OUTFILE)
    @echo done

make hackmd2pdf HACKMD_CODE=bfScafhcTdGzzhJ4B3evlQ

BOOM!¹²

Now, I do understand that from a pure aesthetic point of view this might not be enough to satisfy the sensitive eye of the graphic designer, but it can be shipped!

some dependencies need to be installed, on ubuntu texlive-xetex and pandoc ↩
this command will not work for you, because it contains specific branding stuff from Ape Unit, but if you are looking for a simpler one, look no further! ↩

GitHub is now free for teams 🤔

Andrea Giacobino — Tue, 14 Apr 2020 21:59:46 +0000

Today GitHub announced that you don't need a payed plan anymore to have private repositories for teams.
That is a big announcement and the first thought goes to the competing service, GitLab, that has been offering the same since the very beginning.

But this is somewhat also an interesting announcement that might have implications for the open source movement in general. Of course this is just a pure speculation, but anyway let's roll with it.

I think GitHub has been one of the organization that made the "Free for open source" business model popular. The "Free for open source" model works like this: you can use our service free of charge and we get back advertisement and feed our WOM marketing. Now developers (and companies) tend to be jealous/shy/protective toward their code, but they are also a pragmatical bunch: subscribing to a payed plan for my weekend-long side project or my hello world flutter might not be something worth doing.
But one thing about open source is that, like most thing, is a quantitative issue more then a qualitative one: once you get used to have your stuff open source, it becomes quite liberating (and, who knows, could even help you land a job). We can say that GitHub with its model has, at the very least, encouraged and promoted the open source software experience.

Microsoft on the other hand has historically been the fiercest enemy of the Open Source Movement. Microsoft, since 2018, owns GitHub, and, with this last move, if you are one of those suspicious people, you might think that they kind of want you not to think too much about all this OSS business: get your free proprietary repository and off you go.

Also this.

A pensar male si fa peccato, ma spesso ci si azzecca. - Giulio Andreotti

Self-hosted Jitsi server with authentication

Andrea Giacobino — Tue, 17 Mar 2020 22:24:44 +0000

March 2021 update: the tutorial is still valid for the current Jitsi version but there are some new settings in the config file that are not up to date

As the Coronavirus spreads and more and more people are spending time at home, one of the most critical thing for many is to be able to keep in touch with family, friends and colleagues.

We at Ape Unit are used to remote and distributed work but the current situation meant that it was necessary to go above and beyond....

As we are involved in open source projects and since we already use Jitsi we opted to deploy an instance of Jitsi to use internally and since there have been a lot of demand on the Jitsi community forum about installation and configuration here is a step by step tutorial about how to run your own server.

Once you have completed the tutorial you will have your own Jisti server with authentication enabled.

Before you start

This tutorial assumes that you have basic knowledge about technical stuff in particular assumes that you are familiar with the linux shell, ssh, and dns.

The costs

The running cost of the setup varies depending on the server choice and the domain, in this case we'll be using a Hetzner cloud instance that starts from 2.49€/m

this being said, let's start!

#1. Setup a new server (with Hetzner)

go to the Cloud home
choose the project you want (usually Default)
Click on ADD SERVER

In the Create a Server page make sure to

select the Ubuntu 20.04 as Image (point 2)
add your public ssh key (point 7)

Then click on CREATE & BUY NOW

If you add your ssh key, make sure that it is enable in the ssh configuration of your machine (more on that here). If you do not add your ssh key, you'll receive an email with the root password to the server that you will have to change on first login. As a side note, public key authentication is preferable then password.

The server creation is going to take a minute or less, once completed take note of the IP address of the server.

#2. Point a domain to the newly created server

for this step, we'll assume that you already have a domain registered (you can grab one on namecheap. For this tutorial we will use the domain apeunit.test for our domain.

In your domain DNS configuration add the records:

type: A
host: @
value: IP Address (from the Hetzner server)

NOTE: it might take a few minutes for the DNS to propagate

#3. Jitsi installation

Now that the server is up and running, let's set it up!
If everything before has been successful you should be able to login to your server using:

$ ssh root@apeunit.test
The authenticity of host 'apeunit.test (10.0.0.1)' can't be established.
ECDSA key fingerprint is SHA256:Q1rLmH7vuBalRJGv7sasTJy+ZtS3yOf4A34artGjUI.
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added 'apeunit.test,10.0.0.1' (ECDSA) to the list of known hosts.
Welcome to Ubuntu 18.04.4 LTS (GNU/Linux 4.15.0-91-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage
root@server:~#

We can now proceed with Jitsi installation (documentation here)

First let's make sure that the system is up to date

apt update
apt upgrade -y

Then install nginx

apt install nginx -y

Now we can add the Jitsi repositories and install the Jitsi components. During the installation, you will be prompted for a domain to use, when it happens enter the domain you choose/registered at #2. The second question you'll be asked is if generate a self signed certificate or provide your own; choose the default (generate a self signed certificate)

echo 'deb https://download.jitsi.org stable/' >> /etc/apt/sources.list.d/jitsi-stable.list
wget -qO -  https://download.jitsi.org/jitsi-key.gpg.key | apt-key add -
apt update && apt install jitsi-meet -y

And finally, lets ask let's encrypt for a certificate for our domain

/usr/share/jitsi-meet/scripts/install-letsencrypt-cert.sh

To verify that the installation was successful, open your browser and enter your domain in the address bar, Jitsi should be now up and running.

Once we have verified that the installation is working properly, it is time to configure the firewall so not to leave your server exposed. For firewall we are going to use ufw:

ufw allow ssh
ufw allow http 
ufw allow https 
ufw allow 10000/udp
ufw enable

#4 Enable authentication for Jitsi

Enabling the authentication requires only to edit three files:

/etc/jitsi/meet/apeunit.test-config.js
/etc/prosody/conf.avail/apeunit.test.cfg.lua
/etc/jitsi/jicofo/sip-communicator.properties

NOTE: for the first two, change the apeunit.test part of the filename with your domain

The official documentation for secure domain explains the details of the changes, but here are the files already configured:

I've published the changed files here:

but they have to be changed according to your domain name, here is how. First set a variable with your domain name:

export JITSI_DOMAIN="your_domain.com"

then update the meet configuration file:

curl  https://gist.githubusercontent.com/noandrea/5ff6b212273af95103996c0e71f0cdf2/raw/d14979d5b31af324c5823b3d3c8683a6c762f518/apeunit.test-config.js -s | \
sed  "s/apeunit.test/$JITSI_DOMAIN/g" \
> /etc/jitsi/meet/$JITSI_DOMAIN-config.js

then the prosody configuration file:

curl https://gist.githubusercontent.com/noandrea/5ff6b212273af95103996c0e71f0cdf2/raw/d14979d5b31af324c5823b3d3c8683a6c762f518/apeunit.test.cfg.lua -s | \
sed  "s/apeunit.test/$JITSI_DOMAIN/g" | \
sed  "s/JICOFO_SECRET/$(grep -e '^JICOFO_SECRET=.*' /etc/jitsi/jicofo/config | cut -d '=' -f2)/g" | \
sed  "s/TURN_SECRET/$(< /dev/urandom tr -dc _A-Z-a-z-0-9 | head -c${1:-8})/g" \
> /etc/prosody/conf.avail/$JITSI_DOMAIN.cfg.lua

and finally the sip communicator properties file:

echo "org.jitsi.jicofo.auth.URL=XMPP:$JITSI_DOMAIN" \
>> /etc/jitsi/jicofo/sip-communicator.properties

Once this part is completed it is now time to create your users, with the command prosodyctl register USER DOMAIN PASSWORD; in our example we can register the user jane with:

prosodyctl register jane $JITSI_DOMAIN janepwd

Currently there is no option to make users register themselves, you will have to create users by command line. Also not all the users need to have a password, only the ones that open a room.

To apply the changes last thing we have to do is to restart Jitsi services:

systemctl restart jicofo
systemctl restart jitsi-videobridge2
systemctl restart prosody

That's it! Now you can enjoy your private chat server thanks to the wonderful Jitsi!