DEV Community: Joel Jucá

Meet entr, the standalone file watcher

Joel Jucá — Thu, 02 Nov 2023 15:23:36 +0000

I like to write about tools in my toolbelt I love, and entr is one of these preferred tools I enjoy using a lot.

Today I'm talking about entr – the Standalone File Watcher.

What's `entr`?

entr ("Event Notify Test Runner"; GitHub), is a command-line tool written by Eric Radman that allows running arbitrary commands whenever files change.

It's not tied to any programming language, framework, platform, OS, etc. – it's a real standalone tool, pretty useful in a variety of scenarios.

Usage

First, use find to craft a command that lists the files you're working with:

find . -name '*.md'

Then, just pipe it to entr with a command to be run whenever files change:

find . -name '*.md' | entr echo "Markdown files changed!"

entr has some very interesting options, like the ability to clear the screen before running the given command (so you get a cleaner output). To get a full list of options, check its man page with:

man entr

Continuously testing with entr

As you might have guessed, one of the main use cases for entr is to rerun tests whenever files change. I'm an Elixir engineer, and I use entr to run mix test continuously whenever I save an Elixir file.

This is the snippet I use to continuously test my Elixir library SwissSchema while I'm working on it:

find -E lib test -regex '.*exs?$' | entr -c mix test

A snippet I obviously can't recall from the top of my head every time I need to run it. Luckily, this project has a Makefile with a task for it:

test.watch:
    find -E lib test -regex .*exs?$ | entr -c mix test

💡 I wrote two articles here on DEV about Makefiles, one covering their overall utility and usage, and another with my Makefile setup for Elixir projects:

Makefiles: a standard for project tasks

My Makefile for Elixir/Phoenix

That's it. There's not much to learn, entr is really this awesome!

My Makefile for Elixir/Phoenix

Joel Jucá — Sat, 20 May 2023 15:01:06 +0000

Following my last post about GNU Make and Makefiles, I thought I could share the Makefile I'm using in my Elixir/Phoenix projects to have a standard CLI interface to manipulate all of them.

I designed this Makefile to be easily adapted to be used in other projects (I've been coding with some uncommon tools – Bash, Hy, Janet, Python, SQLite, etc., so remembering the commands specific to each of these is hard. So a standard Makefile helps a lot. I also copy-paste it in new projects, edit its tasks, possibly delete some (eg.: delete database-related commands if they're not needed), and I'm good to go.

If you're not familiar with Makefiles, check this quick introduction to Makefiles I wrote last week. It might give you good insights about what GNU Make is and how it can be used for.

With no ceremony, here's my "standard" Elixir/Phoenix Makefile:

.PHONY: setup build run lint fmt test test.watch db.setup db.create db.migrate db.seed db.drop db.reset repl repl.run repl.test

# Project-wide

setup:
    make build \
    && make db.setup

build:
    mix do deps.get + compile

run:
    mix phx.server

# Code quality

lint:
    mix format --check-formatted

fmt:
    mix format

test:
    mix test

test.watch:
    mix test.watch

# Database

db.setup:
    make db.create \
    && make db.migrate \
    && make db.seed

db.create:
    mix ecto.create

db.migrate:
    mix ecto.migrate

db.seed:
    mix run priv/repo/seeds.exs

db.drop:
    mix ecto.drop

db.reset:
    make db.drop \
    && make db.setup

# REPL

repl:
    iex -S mix

repl.run:
    iex -S mix phx.server

repl.test:
    iex -S mix test.watch

Most of these Mix subcommands are provided by Phoenix, so they work for any Phoenix project you adopt it, with two exceptions: test.watch and repl.test. Both assume that you're using mix_test_watch to run tests every time a change in the source code is made. Everything else is pretty much bare Phoenix subcommands.

That's it. An autocompletion-powered CLI wrapper to work with Phoenix. Now you can type make, hit Tab (once or twice, depending on how your shell is configured), and get autocompletions for the most important tasks of your project.

Makefiles: a standard for project tasks

Joel Jucá — Sat, 13 May 2023 16:57:20 +0000

I've been using Makefiles to create a standard CLI interface for all my projects, no matter what programming language and/or frameworks I'm using.

Makefiles are configuration files for GNU Make that define make tasks for a project (eg.: build, test, etc.).

I have projects in Bash, Elixir, Python, JavaScript, etc., and each of these requires different commands to build, test, run, etc., so I've been using Makefiles to standardize how I run these tasks.

Below is a basic Makefile template I've been using:

.PHONY: build run test

build:    
    # here goes the command(s) to build the project

run:
    # here goes the command(s) to run the project

test:
    # here goes the command(s) to test the project

If your project deals with databases, no matter which one (Postgres, MySQL, SQLite, etc.), you might want to include these tasks too:

.PHONY: db.setup db.reset db.create db.migrate db.seed db.drop

db.setup:
    make db.create \
    && make db.migrate \
    && make db.seed

db.reset:
    make db.drop \
    && make db.setup

db.create:
    # here goes the command(s) to create the database

db.migrate:
    # here goes the command(s) to migrate the database

db.seed:
    # here goes the command(s) to seed the database

db.drop:
    # here goes the command(s) to drop the database

This way your projects can have a "standard CLI interface" for common tasks like building, testing, running, or common database operations like creating and dropping, migrating the schema, etc.

A bit of historical context from GNU Make

Historically speaking, make was used to build files, which often meant compiling source code. Each task was actually a group of commands needed to build a given file if it wasn't present. If the file were in place, the commands wouldn't be executed.

Imagine a JavaScript project that has the following files:

api-sdk.js (functions to use the back-end APIs)
utils.js (general utility functions)
app.js (the main web app code: routes, pages, etc.)

In webdev, it's a common practice to bundle JS files before deploying them to production, so users get to download fewer files, etc., so this project's Makefile could look like this:

bundle.js:
    cat utils.js api-sdk.js app.js > bundle.js

These commands would concatenate all JS source code in these files into one bundle.js. So, this file could be built with the following command:

$ make bundle.js

If the file is absent, Make will run commands to generate it. But then, if you tried to run this command a second time, you would see the following result:

$ make bundle.js
make: `bundle.js' is up to date.

No errors, just this message – but no commands were executed. That's because, by default, Make builds files, and if these are present there's no need to build them again.

However, it's possible to remove this dependency of a file's absence by using the .PHONY special target. You basically add all tasks not tied to a file presence or absence, separated by spaces:

.PHONY: bundle.js

bundle.js:
    cat utils.js api-sdk.js app.js > bundle.js

This way, Make would consistently execute the following commands when make bundle.js is run, no matter if bundle.js already exists or not. It might be useful during development, for instance, to rebuild the file with updated content!

So, the .PHONY special target is generally something you want to use. Just keep it somewhere in your Makefile with a list of all your Make tasks that are file-independent, and it should be good to go!

TIL: Python virtual environments and venv

Joel Jucá — Fri, 03 Feb 2023 21:31:58 +0000

I'm learning Python. The reason is, well, Python is the lingua franca of the Data Science world, and I intend to work with it, eventually. I'm also an old school programmer, so I'm not convinced that "just use X" is the right approach. I refuse to just rely on a Jupyter notebook like Google Collab, etc., so I got to work with Python in real world (which means CLI, Python packages/dependencies, and virtual environments).

At first sight, virtual environments were super confusing! The name "virtual environment" doesn't help understanding what it is. Plus, there are so many ways to set up virtual environments that I was quite lost between all the names and projects, libraries and techniques, etc. Yeah, there was some friction to truly understand it – and I'm still on my way to learn it all.

With it said, I'm sticking with Python's built-in virtual environment tool for now, venv. So the rest of this post assumes venv is being used to work with virtual environments.

WTF are virtual environments anyway?

So far, a virtual environment seem to be a command-line trick to isolate a project from each other. The Python packaging doesn't seem to assume the concept of local package installation (eg.: Node.js' node_modules), it keeps packages into either a global, system-wide scope, or a local, user-wide scope (each system user ending up with its own set of Python package installs).

The problem is that to work on multiple Python projects using your own OS user, you would have conflicts with package versions, just like how you would with a global package scope. :(

So, virtual environments solve this issue. It tricks both Python and pip to install and load packages into a local directory (aka.: your project's directory) instead of global-wide or user-wide directories. It allows you to work on different projects with the same dependencies but different versions without conflicts.

That's it. A quite common approach of pretty much any modern programming language today.

How to use venv

venv is a built-in Python module that does exactly this: manage virtual environments. It is a subset of virtualenv, a Python module that used to be the standard way to set up virtual environments but now compete with other options, like venv.

Using it is really very simple. venv is built-in, so you don't even need to install it, it's already available for you if you're using Python v3.3 or newer.

To create a virtual environment, run:

python3 -m venv .venv

It will create a virtual environment into the .venv directory. You can name it however you like – but .venv seems to be a popular choice, so I'm sticking to it.

Then, in order to use it, you must activate it first:

source .venv/bin/activate

Here's the magic. When you activate the virtual environment, it will trick python, python3, pip, and pip3 to install and load packages from/to .venv. So, each project will have this separation of dependencies. :-)

After you've activated your virtual environment, you can install your local, project-specific dependencies with pip, possibly with the help of a requirements.txt:

pip3 install -r requirements.txt

And when you're done working with this project for the day, you can deactivate (exit from) the virtual environment with:

deactivate

TILs

venv is quite easy to use and work with. It does not isolate runtime versions, though. It seem there are some other projects to do it, but I didn't get to them just yet.

Also, there were a few things that really confused me along the way. For instance, Python literacy talks about virtual environments as something that "isolates Python projects with everything – including Python, pip, etc.", and from it I understood that it somehow could embed a specific Python runtime too. After reading a ton of things online, discussing with people in IRC and Telegram chats, I finally understood that venv does not embed specific Python runtime versions, just dependencies.

The next thing I want to try is virtualenv, which seem more feature-rich. I'll post my findings here.

Thanks for reading!

Test your SSH-based Git auth

Joel Jucá — Mon, 31 Oct 2022 22:22:34 +0000

A small trick I've always used but I realized it might not be a well-known thing.

It is possible to test your authentication against a Git service over SSH with a very simple command:

ssh -T git@your-git-service.com

That's it. Pretty simple, right!? I do it all the time, to test if my SSH key is properly configured on a given Git service (GitHub, Gitea, sourcehut, etc.)

I'll put here some examples for popular Git services, so you can test it easily:

GitHub

$ ssh -T git@github.com
Hi joeljuca! You've successfully authenticated, but GitHub does not provide shell access.

GitLab

ssh -T git@gitlab.com
Welcome to GitLab, @joeljuca!

Bitbucket

ssh -T git@bitbucket.org
authenticated via ssh key.

You can use git to connect to Bitbucket. Shell access is disabled

Gitea

ssh -T git@gitea.com
Hi there, joeljuca! You've successfully authenticated with the key named [key name here], but Gitea does not provide shell access.
If this is unexpected, please log in with password and setup Gitea under another user.

sourcehut

ssh -T git@git.sr.ht
Hi joeljuca! You've successfully authenticated, but I do not provide an interactive shell. Bye!

That's it :) just a small Git/SSH trick for today. I hope it saves you some seconds when you're about to test your SSH-based Git authentication again!

Pro-tip: you can use the verbose mode to look way more hackish if you're testing the authentication in front of your girlfriend. 😂 Try running ssh -Tv instead of ssh -T. 😉

Introducing .iex.local.exs

Joel Jucá — Wed, 07 Apr 2021 05:42:02 +0000

I love command-line interfaces! And, I like to customize my CLI experience as well. Either it being one-character-long aliases, utility functions, or just preloading some data into helper variables, my shell is always customized to some extend.

When I'm working with Elixir, I use IEx, which in turn allows you to customize itself through .iex.exs files. I truly love it! However, if I'm working on a team with other developers and they do like to do the same, we'll be in trouble. If this .iex.exs file is being controlled by Git, we'll both have the exact same customizations (and it won't be customizations anymore).

This week I wrote a solution that's so simple I decided to share.

Behold `.iex.local.exs`!

OK, I'm being a bit extravagant here. 😂 The idea behind this .iex.local.exs file is: any team-wide IEx customization goes on .iex.exs directly (e.g.: customizations that you as a team agreed to use), and anything too personal to be team-wide goes on .iex.local.exs. Then, your .iex.exs file will end with the command below:

# Local dot-iex file (user/environment-specific, Git-ignored)
import_file_if_available(".iex.local.exs")

The file .iex.local.exs should be Git-ignored, so it will be entirely user/environment-specific. There you can create your own aliases, preload/prepopulate utility variables, provide utility functions, etc. You name it!

How about you? What have you been doing/using to be more productive in your work?

Pst! Hey, come here. I have LSD 😎

Joel Jucá — Mon, 14 Dec 2020 15:11:55 +0000

No, you didn't read it wrong. I'm here to tell you about LSD - how cool it is, how to use it, and how to incorporate some awesomeness in your life. Once you taste it, you'll never be the same anymore!

LSD? WTF?

LSD is a Rust-powered ls-like command to list directories and files in your terminal. It works pretty much like ls and can be used as a drop-in replacement, as most of the commonly used options are supported.

The cool thing about LSD is that it improves the file/directory listing experience by cleverly coloring its output, showing some icons beside names, and also providing some interesting customization parameters.

You can just use lsd where you would normally use ls. You could also alias ls to lsd and continue to use it as you normally would:

alias ls="lsd"

Note: ls contains far more options than LSD, and probably should be used when writing shell scripts. Otherwise, for daily work/routine commands, LSD should be safe enough.

My experience with LSD

I've been using LSD for some days now and the experience is good. I feel the file icons and the relative dates help me a lot to find the exact file I'm looking for when I ls somewhere.

Here's my ~/.zshrc alias:

alias ls="lsd --date=relative --group-dirs=first --size=short"

I also wrapped this alias into a which expression that checks if I have the lsd binary available in my $PATH, to avoid errors when using my .zshrc in environments where LSD is not installed/available. So my alias became like this:

# https://github.com/Peltoche/lsd
which lsd > /dev/null 2>&1 && alias ls="lsd --date=relative --group-dirs=first --size=short"

Using LSD

Using LSD is literally the same as using ls. I'll avoid placing installation instructions here as it might become outdated.

The LSD home page is https://github.com/Peltoche/lsd

As of today, the latest LSD version is 0.19.0. You can check all its optional parameters through --help:

$ lsd --help
lsd 0.19.0
An ls command with a lot of pretty colors and some other stuff.

USAGE:
    lsd [FLAGS] [OPTIONS] [--] [FILE]...

FLAGS:
    -a, --all               Do not ignore entries starting with .
    -A, --almost-all        Do not list implied . and ..
        --classic           Enable classic mode (no colors or icons)
    -L, --dereference       When showing file information for a symbolic link, show information for the file the link references rather than for the link itself
    -d, --directory-only    Display directories themselves, and not their contents (recursively when used with --tree)
    -X, --extensionsort     Sort by file extension
        --help              Prints help information
    -h, --human-readable    For ls compatibility purposes ONLY, currently set by default
        --ignore-config     Ignore the configuration file
    -F, --classify          Append indicator (one of */=>@|) at the end of the file names
    -i, --inode             Display the index number of each file
    -l, --long              Display extended file metadata as a table
        --no-symlink        Do not display symlink target
    -1, --oneline           Display one entry per line
    -R, --recursive         Recurse into directories
    -r, --reverse           Reverse the order of the sort
    -S, --sizesort          Sort by size
    -t, --timesort          Sort by time modified
        --total-size        Display the total size of directories
        --tree              Recurse into directories and present the result as a tree
    -V, --version           Prints version information
    -v, --versionsort       Natural sort of (version) numbers within text

OPTIONS:
        --blocks <blocks>...            Specify the blocks that will be displayed and in what order [possible values: permission, user, group, size, date, name, inode]
        --color <color>...              When to use terminal colours [default: auto]  [possible values: always, auto, never]
        --date <date>...                How to display date [possible values: date, relative, +date-time-format] [default: date]
        --depth <num>...                Stop recursing into directories after reaching specified depth
        --group-dirs <group-dirs>...    Sort the directories then the files [default: none]  [possible values: none, first, last]
        --icon <icon>...                When to print the icons [default: auto]  [possible values: always, auto, never]
        --icon-theme <icon-theme>...    Whether to use fancy or unicode icons [default: fancy]  [possible values: fancy, unicode]
    -I, --ignore-glob <pattern>...      Do not display files/directories with names matching the glob pattern(s). More than one can be specified by repeating the argument [default: ]
        --size <size>...                How to display size [default: default]  [possible values: default, short, bytes]
        --sort <WORD>...                sort by WORD instead of name [possible values: size, time, version, extension]

ARGS:
    <FILE>...     [default: .]

If you use LSD and enjoy it, let me know how awesome it is for you in the comments below. 😉

The ES6 in Depth Challenge

Joel Jucá — Mon, 21 Sep 2020 11:04:38 +0000

Mozilla (the creators of Firefox and MDN) has this interesting blog called Mozilla Hacks where they post interesting advanced tricks with Web technologies. In April of 2015, they started a series of blog posts entitled ES6 in Depth.

It's basically a list of articles, one for each JS feature coming on ES6: let and const, classes, for-of and iterators, modules, etc., with a decent dive into the details. It's a great way to learn post-2015 modern JavaScript.

Challenge Time!

The challenge is simple: read one ES6 in Depth article every two days. You'll read the article and spend two days understanding and using it, incorporating it into your toolbelt, and making yourself familiar with it. Then, on the third day, read the next article and start over again - until you read them all.

It's a series of 17 blog posts - being 15 about features, an introduction, and a final post. Spend two days on each feature article and you'll finish it in one month. After that, you'll know and have experienced much of today's JS, definitely leveling up your career as a developer.

Rules

In order to make the best out of this experience, you'll follow the rules below:

#1 Don't rush

The challenge is about absorbing knowledge and some experience. It takes time, often way more than two days. So don't rush. Becoming a better developer takes time, so give yourself time to think about how these features can help you to write better code.

#2 Don't give up

If you miss a day or two for some important reason, don't give up. Reorganize your daily routine and move on. Start the cycle again the next day and keep going until you read all articles.

Don't be sorry if you're falling behind, don't get distracted, don't give a chance to bad feelings.

Accomplishing things is hard, so if you fall on this road, you get up and keep going. Once you finish all the articles you'll feel great for accomplishing it, and better: you'll profit from this knowledge for the rest of your career.

# Keep a log

Write down the things that you're learning somewhere. It can be a note-taking app, a blog, email drafts, whatever. Writing will help you digest knowledge much better than just reading, so write down what you've learned, along with examples, ideas, notes, snippets, etc.

These writings might become your own blog posts in the future. :-)

🙂

Do you think that challenges like this one can inspire and motivate you to learn something? Let me know how useful it is for you in the discussions below!

I'm always on Twitter as @holyshtjoe. If you want to discuss programming, you know how to find me. 😉

git standup

Joel Jucá — Wed, 16 Sep 2020 18:46:19 +0000

It's Daily Standup time and there you are: unsure about what you did since yesterday - not because you did nothing but because you forget things easily. 😂

Yeah, the person here is me, and I often can't answer quickly when I'm asked "What have you done today?". Well, fortunately, I'm a programmer who knows Git, and I made a command that quickly reminds me of what I've done since yesterday:

git standup

Cool, right? Yeah, this is a Git alias, a shortcut for a more complex Git command. Git aliases reside on a file named .gitconfig in your home folder. To set up my own git standup alias, I added the following line to ~/.gitconfig:

[alias]
  standup = log --all --since yesterday --author=joel

This aliased Git command will list all commits from all branches that have been made in the last 24 hours by an author that contains joel somewhere on his name or email.

You might want to change the author field with something unique to your name/email.

Note: if you already have a [alias] line on your .gitconfig there's no need to add another one. Just make sure your standup alias is placed after it.

This is it. After saving the file, you'll be able to use your alias as a Git subcommand:

git standup

Pro tip: since I tend to prefer shorter outputs, I empowered my git standup with some additional flags:

[alias]
standup = log --all --abbrev-commit --graph --pretty=oneline --since yesterday --author=joel

Now my git standup outputs something like this:

$ git standup
7b2280e (origin/branch-123) I did something awesome here
7b2280e (branch-123) Here goes a hidden bug packed as a feature to production
15f61fa (refs/stash) WIP
433ed2b index on branch-456: e6f9335 This is another great commit

PS: if you're curious about how my .gitconfig looks like, it's public on my GitHub. You can check it here.

How about you? Do you have any Git trick that makes your life as a developer easier? Share it in the discussions below. 🙂