DEV Community: Ema Suriano

Get Github repository preview images

Ema Suriano — Thu, 26 Sep 2024 00:00:00 +0000

While I was writing another TIL I needed to put some image about an open source project, in this case, was the popular project of Ollama. Normally what I do is take a screenshot of the website and store it locally in my assets folder, which is what most of the websites also do.

The main issue of this problem is not the fact that the repository size is going to grow (I solved this issue by using Git submodules), it is that the image most probably is going to get out of date. The information about the post is most probably also going to get outdated, but having as well old pictures doesn't help a lot ...

While I was searching for some kind of image that represents the project without information that can get outdated, I found this preview:

It caught my attention because all the information was extremely up-to-date, considering that the last commit was less than 12 hours ago.

Then I decided to inspect the URL and, I found out that, it was coming from the Open Graph of Github assets. The URL itself looks like:

https://opengraph.githubassets.com/16dc0ce438cf338ca03e98e84238f23161d058f1665111075ba68fcd2f0ecf04/ollama/ollama

Then I checked on the Internet what each part means, and I found the following explanation:

https://opengraph.githubassets.com/<random_hash>/<owner>/<repo>

After knowing this I started playing with some of my other repositories and I found that I can always reliably get an up-to-date Github preview of my projects. As an example, I generated the one for this website:

That will be all for now.

Thanks for reading.

Running local LLM is not as hard as I expected

Ema Suriano — Thu, 26 Sep 2024 00:00:00 +0000

Let's keep it short, if you don't have a dedicated GPU you can still run locally an LLM and keep your data private to you, or even be creative and integrate it into an application or your workflow without having to pay per generated token (which is the way people pay for the closed-source LLM).

The open-source project that will make your life easier is called Ollama.

Ollama can be described as an LLM manager, with which you can easily install and run models locally on your computer.

The installation is pretty straightforward forward and it's now officially supported for all the Operative Systems: macOS, Linus, and Windows.

After the installation is over the way you interact with Ollama is via the Terminal application, using the command ollama.

➜ ~ ollama
Usage:
  ollama [flags]
  ollama [command]

Available Commands:
  serve Start ollama
  create Create a model from a Modelfile
  show Show information for a model
  run Run a model
  stop Stop a running model
  pull Pull a model from a registry
  push Push a model to a registry
  list List models
  ps List running models
  cp Copy a model
  rm Remove a model
  help Help about any command

Flags:
  -h, --help help for ollama
  -v, --version Show version information

Use "ollama [command] --help" for more information about a command.

All the models you can install and run are listed on their official website on the Library page. They are by default listed as "Featured", which tends to be from better to worse (but that can change).

The most popular model to run with Ollama is Llama from Facebook. At the moment of writing this article, they just announced their new model llama3.2.

To install and run it, simply execute:

➜ ~ ollama run llama3.2

The first time you run it, it will download the model and once everything is done you are going to see a prompt in your terminal saying "Send a message". This is the simplest way you can try the model and see how the responses are.

>>> Hey what's up?
Not much! It's nice to meet you. Is there something I can help you with or
would you like to chat?

>>> I'm writing a post, can you send a short message to the readers :)
Here is a short and sweet message:

"Thank you for being here!"

Feel free to use it as is or modify it to fit your post's tone and style!

Going a step further

We can all agree that using the terminal to communicate with the model can be a bit cumbersome. Compared to other LLM experiences like Chat GPT, Claude, or even Bing, we are missing many features, such as History, the possibility of uploading images, file upload, sending audio, etc.

This is the problem that Open WebUI is coming to solve!

It's a project that runs on top of Ollama and allows you to interact with the models in many ways. You can see it as a front end of your Backend (Ollama).

They ship tons of features so I highly recommend checking their websites where they explain all of them: Features.

To install the project the easiest way is to download the Docker image that gets automatically updated when they release a new version. For this, you have to install Docker on your machine if you don't have it yet.

Then execute this command:

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

This will download the image, install the required dependencies in the image, and start the daemon instance, which you can easily access by entering https://localhost:3000.

The first time you are going to see a Login screen, just create an account. This is in case you want to host the server and would like to restrict access or split the history between the users.

And now you can fully enjoy the full experience :)

Thanks for reading.

Remind yourself of Birthdays Using Google Calendar and Apps Script

Ema Suriano — Mon, 05 Aug 2024 00:00:00 +0000

If you keep track of friends' and family members' birthdays using Google Contacts, you can receive automatic email notifications on the day of their birthdays by integrating Google Calendar and Google Apps Script. Here’s how you can set it up:

The first thing to do is to enable the Birthday Calendar. This can be done by enabling the calendar "Birthday" in Google Calendar, and you should see a calendar named "Contacts" inside "Other Calendars".

Once turned on you should see the name of your contact along with a cake emojii (🎂), and they should be automatically repeated every year. This is the integration between Google Calendar and Google Contacts.

The last thing we need before making the automation is to obtain the Calendar ID. For that click the three dots next to the "Birthdays" calendar, go to Settings and sharing, and find the Calendar ID. It might look like #contacts@group.v.calendar.google.com.

With that we can create our small script, so let's open Google Apps Script through script.google.com or from any Google Sheet by navigating to Extensions > Apps Script. Use the following script to check for birthdays and send email reminders:

var CALENDAR_ID = "#contacts@group.v.calendar.google.com"; // Use your Birthday calendar ID
var EMAIL = "your.email@example.com"; // Change this to your email

function sendBirthdayReminders() {
  var today = new Date();
  var events = CalendarApp.getCalendarById(CALENDAR_ID).getEventsForDay(today);

  events.forEach(function (event) {
    var title = event.getTitle();

    Logger.log("Sending email for event: " + title);
    MailApp.sendEmail({
      to: EMAIL,
      subject: "Birthday Reminder: " + title,
      body:
        "Today is " + title + "'s birthday! Don't forget to wish them well.",
    });
  });
}

You can run the script manually to ensure it’s working as expected. Make sure to authorize any permissions for the script to access your Calendar and Gmail.

Lastly, we need to set up an automatic trigger to run the script every day. In the Apps Script editor, click on the clock icon to set up a trigger. Add a new time-driven trigger to run sendBirthdayReminders daily at a specific time (e.g., 7:00 AM).

Bonus Tip : You can potentially customize the script to include special messages or even send reminders to multiple people if needed!

Thanks for reading.

How to Install Fuzzy CLIPS: A Quick and Easy Setup Tutorial

Ema Suriano — Wed, 31 Jul 2024 00:00:00 +0000

Fuzzy CLIPS is an extension of CLIPS (C Language Integrated Production System) designed for handling fuzzy logic. It allows systems to make decisions based on approximate or imprecise information.

Fuzzy CLIPS is an older tool with limited and outdated online documentation. This is the main reason I'm writing this short post.

Why Fuzzy Logic?

Fuzzy logic excels at handling real-world complexity and imprecision, it allows you to model human-like reasoning, deal with ambiguous data, and create more intuitive control systems. By using degrees of truth instead of binary (true/false) decisions, fuzzy logic can provide smoother, more accurate results in situations where traditional logic falls short

It's precious in fields like control engineering, decision support systems, and AI, which can lead to more robust and flexible solutions that better mimic human decision-making processes.

Installing Fuzzy CLIPS

The source code can be found in GitHub but as expected, this cannot be directly executed in our system. We need to compile the program to run it locally.

Let's start by cloning the repository:

$ git clone https://github.com/rorchard/FuzzyCLIPS && cd FuzzyCLIPS/source

Now we are going to make use of the make command which is available by default in OSX and Linux. In case you are running Windows, please check to install using choco (reference).

$ make fzclips

This should create an executable binary called fzclips which we can execute:

$ ./fz_clips

FuzzyCLIPS V6.10d (10/22/2004)
FuzzyCLIPS>

Fuzzy Concepts

To understand how Fuzzy Clips works, is better to do a small recap of some of the core concepts of Fuzzy Logic.

Fuzzy sets : extend classical set theory by allowing partial membership. In a classical set, an element either belongs to the set (1) or doesn't (0). In a fuzzy set, an element can have any value between 0 and 1, indicating its degree of membership.

Example: In a fuzzy set for "tall people," someone who is 5'10" might have a membership degree of 0.7, while someone who is 6'2" might have a membership degree of 0.9.

Linguistic variables : These are variables whose values are words or sentences rather than numbers. They allow us to express complex concepts in natural language terms.

Example: Temperature could be a linguistic variable with values like "cold," "cool," "warm," and "hot."

Fuzzy rules : These are conditional statements that use linguistic variables, typically in an IF-THEN format. They form the basis of fuzzy reasoning.

Example: "IF temperature is hot AND humidity is high THEN comfort is low"

Fuzzification : This is the process of converting crisp (precise) input values into fuzzy values. It involves determining the degree to which these inputs belong to each of the appropriate fuzzy sets.

Example: Converting a precise temperature of 28°C into fuzzy values like "0.7 warm" and "0.3 hot"

Inference : This step applies the fuzzy rules to the fuzzified inputs to determine the fuzzy output. It involves evaluating all rules in parallel and combining their results.

Example: Applying multiple rules about temperature, humidity, and wind to determine the overall comfort level

Defuzzification : This is the final step where the fuzzy output is converted back into a crisp (precise) value. There are several methods for this, such as the centroid method or mean of maximum.

Example: Converting a fuzzy comfort level (e.g., "0.6 uncomfortable") into a specific value on a scale of 1-10

Demo time: Fan Control System

Let's create a fuzzy logic system to control a fan based on temperature and humidity. The system will adjust the fan speed (low, medium, high) depending on the fuzzy input values for temperature and humidity.

Create a new file with the extension .clp and open it with any code editor that you want.

1. Definition of Membership functions

(deftemplate Temperature
  0 40
  ((cold (0 1) (10 0))
   (warm (5 0) (20 1) (30 0))
   (hot (25 0) (40 1) (40 1))))

(deftemplate Humidity
  0 100
  ((low (0 1) (20 0))
   (medium (10 0) (50 1) (90 0))
   (high (70 0) (100 1) (100 1))))

Here, Temperature and Humidity are the fuzzy variables with membership functions cold, warm, hot, low, medium, and high. These are going to be our entry points for the system.

(deftemplate FanSpeed
  0 100
  ((slow (0 1) (30 0))
   (medium (20 0) (50 1) (80 0))
   (fast (60 0) (100 1) (100 1))))

Lastly let's add the FanSpeed with slow, medium, and fast functions, which are going to get activated based on the rules of the system.

2. Definition of rules

Create rules to determine the fan speed based on temperature and humidity:

(defrule cool-fan
  (Temperature cold)
  (Humidity low)
 =>
  (assert (FanSpeed slow)))

(defrule moderate-fan
  (Temperature warm)
  (Humidity medium)
 =>
  (assert (FanSpeed medium)))

(defrule hot-humid-fan
  (Temperature hot)
  (Humidity high)
 =>
  (assert (FanSpeed fast)))

These rules set the fan speed to slow, medium, or fast based on the combination of temperature and humidity.

3. Definition of Facts

Provide sample data for temperature and humidity:

;; Sample facts for temperature and humidity
(deffacts sample-facts
  (Temperature (25 0) (25 1) (25 0))
  (Humidity (40 0) (40 1) (40 0)))

These facts represent a temperature of 25°C and a humidity of 40%.

4. Running the system

Open a new terminal in your system and execute the program fz_clips - the one that we compiled in the beginning. Once running, we need to load our program into the library. For this, we need to use the load function and send the path of the file containing the rules we defined previously:

FuzzyCLIPS> (load ./demo.clp)
Defining deftemplate: Temperature
Defining deftemplate: Humidity
Defining deftemplate: FanSpeed
Defining defrule: cool-fan +j+j
Defining defrule: moderate-fan +j+j
Defining defrule: hot-humid-fan +j+j
Defining deffacts: sample-facts
TRUE

Now we need to initialize the environment, to load all the facts and rules into the memory:

FuzzyCLIPS> (reset)

We can double-check if everything was loaded correctly with:

FuzzyCLIPS> (facts)
f-0 (initial-fact) CF 1.00
f-1 (Temperature ???) CF 1.00
        ( (25.0 0.0) (25.0 1.0) (25.0 0.0) )

f-2 (Humidity ???) CF 1.00
        ( (40.0 0.0) (40.0 1.0) (40.0 0.0) )

For a total of 3 facts.


FuzzyCLIPS> (rules)
cool-fan
moderate-fan
hot-humid-fan
For a total of 3 defrules.

Now it's time to execute the rules based on the asserted facts.

FuzzyCLIPS> (run)

Alternatively, we can make partial execution by specifying the number of steps, for example, rule by rule:

FuzzyCLIPS> (run 1)

At any point, we always check our facts to see the status of our system.

FuzzyCLIPS> (facts)
f-0 (initial-fact) CF 1.00
f-1 (Temperature ???) CF 1.00
        ( (25.0 0.0) (25.0 1.0) (25.0 0.0) )

f-2 (Humidity ???) CF 1.00
        ( (40.0 0.0) (40.0 1.0) (40.0 0.0) )

f-3 (FanSpeed ???) CF 1.00
        ( (20.0 0.0) (35.0 0.5) (65.0 0.5) (80.0 0.0) )

For a total of 4 facts.

As we can see the fact of FanSpeed has been loaded and some values set. Fuzzy clips provides a way to visualize it in the terminal by using the function plot-fuzzy-value:

FuzzyCLIPS> (plot-fuzzy-value t + 0 100 3)

Fuzzy Value: FanSpeed
Linguistic Value: ??? (+)

 1.00
 0.95
 0.90
 0.85
 0.80
 0.75
 0.70
 0.65
 0.60
 0.55
 0.50 +++++++++++++++
 0.45 + +
 0.40 + +
 0.35 + +
 0.30
 0.25 + +
 0.20 + +
 0.15 + +
 0.10
 0.05 + +
 0.00+++++++++++ +++++++++++
     |----|----|----|----|----|----|----|----|----|----|
    0.00 20.00 40.00 60.00 80.00 100.00

Universe of Discourse: From 0.00 to 100.00

Finally, we can perform a defuzzification process on any fact to see its crisp value:

FuzzyCLIPS> (moment-defuzzify 3)
50.0

Fun fact: Clips is compatible with Clojure syntax

In case want to keep a strict formatting in your code, I have good news for you! You can change your syntax language to Clojure and it will format your code accordingly. The main reason is because Clojure tries to keep the same formatting practices as in LISP, therefore it's compatbile with CLIPS as well!

You can easily install any formatter in your code editor and on save you can pass from this:

(deftemplate Humidity
    0 100
    (
        (low (0 1) (20 0))
        (medium (10 0) (50 1) (90 0))
        (high (70 0) (100 1) (100 1))
    )
)

To this

(deftemplate Humidity
  0 100
  ((low (0 1) (20 0))
   (medium (10 0) (50 1) (90 0))
   (high (70 0) (100 1) (100 1))))

Which one is better? I actually don't care so much, what I care is all my code is now formatted in the same way and I don't have to manually start adding spacing.

Another tip, you can use the formatter as some sort of compiler mostly to check if you skip some parenthesis while you were writing. As you can see from the previous example, there quite a lot of them and it can quite easy to forget to close one of them. The formatter will fail in case you have some parenthesis that are not properly closed, which means that you can speed up your process because you don't have to load and execute your program in Fuzzy CLIPS to see the error message.

Closing words

As I mentioned at the beginning of this article, the amount of information about this framework is quite scarce and it took me quite some time to gather all this information. Hopefully, more people will find it handy as well!

Thanks for reading.

Adding support for LaTeX in Astro.js

Ema Suriano — Mon, 24 Jun 2024 00:00:00 +0000

LaTeX can be considered the standard language to represent mathematical expression in any possible field, and adding it to your page content is not that complicated. In case you already have an Astro project, and you are generating your pages from Markdown files, then you can easily add support by extending your Markdown plugins.

The great open source project remark-math allows you to extend your markdown and be able to use the classic $$ or $ and transform them into LaTeX syntax on the spot. The first step is to install both remark-math and one of the math rendering extensions: rehype-katex (uses KaTeX) or rehype-mathjax(uses MathJax).

I personally prefer rehype-katex mostly because the LaTeX text can be selected in the HTML and copied to the clipboard, where rehype-mathjax simply generates an svg tag that doesn't allow selection.

The next step is to add both extensions to your astro.config.mjs:

import { defineConfig } from "astro/config";
import remarkMath from "remark-math";
import rehypeKatex from "rehype-katex";

export default defineConfig({
  ...restConfig,
  markdown: {
    syntaxHighlight: "prism",
    remarkPlugins: [remarkMath],
    rehypePlugins: [rehypeKatex],
  },
});

In case you also decided to use rehype-katex you have to add the following link tag to the head of your page, to load additional styling that KaTeX needs to render properly:

<!-- Katex -->
<link
  rel="stylesheet"
  href="https://cdn.jsdelivr.net/npm/katex@0.15.2/dist/katex.min.css"
  integrity="sha384-MlJdn/WNKDGXveldHDdyRP1R4CTHr3FeuDNfhsLPYrq2t0UBkUdK2jyTnXPEK1NQ"
  crossorigin="anonymous"
/>

The setup of the extensions is now done and you can represent LaTeX by using the $ symbol. Some examples are:

- Inline expression: $f(x) = x + 1$

- New line + centered expression:
  $$
  f(x) = x + 1
  $$

Which looks the following way:

Inline expression: $f(x) = x + 1$
New line + centered expression: $$ f(x) = x + 1 $$

That's all for now, thanks for reading.

Simplifying project navigation with git-open

Ema Suriano — Wed, 19 Jun 2024 00:00:00 +0000

If you are the kind of person that works in different projects at the same time and have a folder of bookmarks with the links to the repositories, then git-open is the CLI for you!

Once installed, the only thing you have to do is run git open from your project folder, and you will see a new browser tab open with the repository linked to it (it takes this information from the origin) with even the branch you are currently working.

To install it simply run:

npm install -g git-open

Because it's installed globally, you can access the tool from any project you are working on. Once you get used to the command, it becomes quite natural to use it to have quick access to PRs or issues.

Oh, it works with many providers:

You can find the official repository in GitHub at: https://github.com/paulirish/git-open

Thanks for reading.

Run a Linux Distro in your Android device

Ema Suriano — Thu, 11 Apr 2024 00:16:49 +0000

This post is going to be about how I set up an Android tablet to be able to run a Linux Distro in order to have a desktop experience and boost the productivity due to the Window Manager.

Disclaimer: The whole process happens locally in the tablet, therefore all the programs are executed inside the tablet. I saw some other tutorials where people use code-server and besides the coding experience might look the same, running the full OS offers more capabilities.

This is how the end result looks like:

How I got here?

While I was travelling and reading some Machine Learning books, I wanted to try some of the new things that I was reading, but sadly I didn't have any laptop with me, and I was not planning in getting a new one. The only big screen that I had with me was a 10" Tablet, which I use for streaming and reading.

The tablet is a Redmi Pad SE running Android 14 with 8 GB of RAM, Snapdragon 680 for the CPU and 256 GB of disk. It's not a bad tablet at all, but clearly is not the best one, specially the CPU could have been better.

The only piece of Hardware that I would highly recommend getting is a Bluetooth keyboard. Using the virtual keyboard in the screen takes quite a lot of space in the screen and highly reduce the productivity while using the desktop. I got a Logitech K380, which is one of the cheapest and most compact Bluetooth keyboard that you can find stores.

One valid argument can be: what's the point of running Linux in a tablet if I already have a computer/laptop for development? I came up with the following points:

Convenience: In case you don't have computer around, then with this setup you can do basically the same stuff. In my case, I tried cloning repositories, installing dependencies, running node server, opening browser and inspecting the website.
Affordability: If you don't have a computer, this setup is way cheaper than buying a new laptop or even worse the whole setup for a desktop computer.
Portability: This point might depend on your Hardware, in my case the combo of Tablet + Keyboard is lighter than a laptop. Also considering that you can use the same charger that you use for your phone.

Stack

The only things applications that you need to download are:

- Termux: terminal application that allows to emulate bash and have access to package installation like apt install.

- Termux:X11: plugin for Termux to connect to instance of servers and handle the connection.

Important disclaimer: You can find the latest version of Termux inside F-Droid instead of Play Store, so please download those instead.

I want to keep this post mostly practical, therefore in case you want to read how the actual desktop emulation run, please check out the official repository of the plugin. It provides an in depth description of the inner commands and options available inside the plugin.

Once both applications are installed, we can start with the installation of the project dependencies and image of the OS. For this step I found a great GitHub repository where the whole process is automatized, so I want to send great kudos to @phoenixbyrd and rest of the team which made the installation script. Inside his README you can find:

Sets up a Termux XFCE desktop and a Debian proot install. This setup uses Termux-X11, the termux-x11 server will be installed, and you will be prompted to allow Termux to install the Android APK.

Also consider that you are going to need at least 4 GB of storage space for the installation and as you install applications, they will take even more.

Open Termux and paste the following command.

> curl -sL https://raw.githubusercontent.com/phoenixbyrd/Termux_XFCE/main/setup.sh -o setup.sh && chmod +x setup.sh && ./setup.sh

This might take some time because it has to download all the resources + packages for the OS. In case you encountered some issue during the installation, you can

join their Discord server or check out the GitHub Issues of the repository.

Assuming your installation is successful, the setup already added a new start command into your Termux that allows you start the server and at the same time boot Termux:X11 with the same port.

I tested several setup in the past using approaches like Andronix and this setup is better in many ways:

Installation: the setup script takes care of downloading the resources you need and a single place for troubleshooting. This is extremely helpful because there are so many parts that can be misconfigured and searching in Stack Overflow is not that helpful due to the configuration of someone might defer from yours.
Smoother experience: Termux:X11 provides a smoother experience where the response time is faster to other alternatives.
One button to start: simply type start in Termux and in less than 5 seconds you are inside your desktop. No need to configure IP or have both application running, the command does everything for us.

Configuration of the Desktop

At this point you can do pretty much whatever you want with the desktop like installation packages or applications. Nevertheless, that you are not running a real Unix Kernel and some modules might be missing, that's why not all the applications can be installed directly.

Luckily this project already have the solution for that! In your desktop there is an app called "App Installer". Once opened you can find many applications that are "modded" so they can run in this kind of virtual environment.

Some good examples are Visual Studio Code (even Extensions works!), Notion, Brave browser, PyCharm and many more! Just select the one you want to install and click on OK.

Small caveat: there is no installation progression indicator, so after selecting the application to install it might seem that nothing is happening, but simply wait a couple of minutes, and you should see a new alert saying that the App has been installed.

Demo time

Honestly anything can be a proof that the setup works well, because at this point you can browse the internet, run bash scripts or even play desktop games. In my case, I use it mostly for web development, so that's what I'm going to showcase.

The first step is to install the IDE to handle files and changes. Nowadays, VS Code is the go-to editor for web development, which can be easily installed via the "App installer" as I described before.

Once installed we can open it and select the option to clone a new repository. I use GitHub to store my public repositories, which VS Code has an integration already built in for it.

Depending on the stack of the repository you are cloning, you might have to install additional dependencies. For this demo, I'm using my own website, which is a static website built with Astro.js. It which requires to have Node.js installed and Yarn for package manager.

Once all the dependencies are installed, we can now start the server. In my case, I run yarn start which starts a local server that can be opened in the browser.

For Web Development one of the most powerful tools is the Dev Tools, where you can inspect DOM Nodes, network activity or even trigger commands from the terminal. This kind of tooling is normally missing in Android and thanks to this setup we can have it.

Thanks that we are running the local server, can we modify the codebase and see the changes in real time inside our browser. In this case, I modify the name and the company.

Once we are happy with the changes, we can commit our changes to saved our changes and potentially trigger a new deployment. With this we basically cover how we can do basic web development.

Closing words

I normally don't write this kind of posts of "tech hacks" but I had so much fun tweaking the tablet that I just wanted to make a post documenting the process to help others. On top of that, when I was re-searching for this kind of stuff I would have wished to find some kind of post like this one!

Feel free to reach out to me or leave a comment in this post, I can try to help with the process or even better you consider joining the Discord server where the real maintainers of the project can help you.

Have fun hacking, and thanks for reading.

Keep your Documentation updated with Cypress and GitHub Actions

Ema Suriano — Tue, 03 Nov 2020 10:01:18 +0000

One of my favorite ways to document projects is by adding screenshots of how it looks, to provide a quick overview of what it does and looks like. Sadly these images are quite easy to get out-dated, and I was being forced to manually update them ... In this post, how I automatize this task by using Cypress and GitHub Actions.

Continue reading on Medium »

Keep your Documentation updated with Cypress and Github Actions

Ema Suriano — Mon, 26 Oct 2020 00:00:00 +0000

My Case in Particular

I am the owner and maintainer of the Gatsby Starter Mate, which allows developers or tech writers to bootstrap their portfolio and manage its content with a CMS.

Although one can always open the Demo website and navigate through it to see how the project is, I decided to create a simple Table where I show how each of the sections looks like. Here is the table extracted from the README.md.

As I highlighted at the beginning of this post, in case I made a change in the UI: CSS change, content change, add/remove new components, etc. I had to regenerate these screenshots ... The process was as follows:

Start the development server.
Open the website in the browser (trying to always use the same window size).
Take a screenshot and store it in the repository.
Remove the old image and rename the new with the original name.

Besides the process was not hard to follow, I had to repeat it for each of the sections and in many times I forgot to do it after I made any change ... It was time to bring the machines to the play! 🦾

Setting up Cypress 🤖

Cypress is a JavaScript End-to-End Testing that provides a nice framework to work with and it's capable of emulating a browser that can interact with any website by using the browser APIs.

Even though this tool was built in mind of Testing you can use it as a Photographer, to make the Browser to take screenshots of your website and store them on your repository!

As this tool is not needed when the application is running in production, it's always recommended installing it as a development dependency:

# npm
> npm install cypress --save-dev

# yarn
> yarn add cypress --dev

Then you have to create a file called cypress.json located at the root of your directory, providing the URL of your project:

{
  "baseUrl": "http://localhost:3000/",
  "screenshotsFolder": "screenshots"
}

DISCLAIMER: In case you provided a localhost route, you have to ensure that your development server is running in the background when Cypress is running.

Next, let's add a simple health check that will navigate to the Home Page of the project and check if the local server is answering correctly. For that create a file called health.spec.js, located inside a folder called cypress/integration at the root folder of the repository:

// cypress/integration/health.spec.js

it('health test', () => {
  cy.visit('/');
});

After this you can finally execute Cypress by running:

> yarn cypress run

====================================================================

  (Run Starting)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Cypress: 5.4.0 │
  │ Browser: Electron 85 (headless) │
  │ Specs: 1 found (health.spec.js) │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

────────────────────────────────────────────────────────────────────────────────────────────────────

  Running: health.spec.js (1 of 1)

  ✓ health test (293ms)

  1 passing (304ms)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests: 1 │
  │ Passing: 1 │
  │ Failing: 0 │
  │ Pending: 0 │
  │ Skipped: 0 │
  │ Screenshots: 0 │
  │ Video: false │
  │ Duration: 0 seconds │
  │ Spec Ran: health.spec.js │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

====================================================================

  (Run Finished)

       Spec Tests Passing Failing Pending Skipped
  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ ✔ health.spec.js 301ms 1 1 - - - │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
    ✔ All specs passed! 301ms 1 1 - - -

✨ Done in 9.67s.

This command will create the folders of plugins and support inside the Cypress folder with some boilerplate inside, which will come in handy for the future steps.

Taking screenshots with Cypress 📸

Time to make Cypress take those screenshots! Let's start by adding a new file inside integration called photographer.test.js, with some basic tests:

describe('Photographer', () => {
  beforeEach(() => {
    cy.viewport('macbook-15'); // Desktop viewport
  });

  // Taking screenshots by Sections
  it('Landing Section', () => {
    cy.visit('/');
    cy.get('#home').screenshot('Landing');
  });

  it('About Section', () => {
    cy.visit('/');
    cy.get('#about').scrollIntoView().screenshot('About');
  });

  // Taking screenshots of the whole page
  it('404 Page', () => {
    cy.visit('/');
    cy.get('#404').screenshot('404');
  });
});

The URL of the pages to visit and CSS selectors may change depending on your project, but the idea will remain the same:

Visit a particular route of your application.
Use a CSS selector to pick a section or grab all the content.
Take a screenshot and save it using a descriptive name.

Once you are done with setting the routes and name of your section, run Cypress by executing yarn cypress run, and you will see a new folder called screenshots located at the root of your project with the screenshots taken by Cypress.

📦screenshots
 ┗ 📂photographer.test.js
   ┣ 🖼404.png
   ┣ 🖼About.png
   ┣ 🖼Landing.png
   ┣ 🖼Projects.png
   ┗ 🖼Writing.png

Be aware that Cypress will remove and recreate this folder on every run, so don't worry if your images disappear between runs. Also, at this point you can remove the test health.spec.js, because it's not needed anymore 👍

The last piece will be changing your documentation to point to the auto generated pictures instead of the manual. I recommend using a table because it provides a very clear interface for the readers to visualize the pictures. This is the one I used:

## Screenshot and Design 🖼

| Section | Screenshot |
| -------- | :--------------------------------------------------------: |
| Home | ![Home](screenshots/photographer.test.js/Landing.png) |
| About me | ![About me](screenshots/photographer.test.js/About.png) |
| Projects | ![Projects](screenshots/photographer.test.js/Projects.png) |
| Writing | ![Writing](screenshots/photographer.test.js/Writing.png) |
| 404 | ![404](screenshots/photographer.test.js/404.png) |

Automatizing the screenshot (now for real)

Up to this point, Cypress is taking the screenshots for me, but I still have the task to call it by running yarn cypress run, and also I have to make sure my development server is running at that time to generate valid screenshots.

To address the latter, I recommend using an npm package called start-server-and-test which does exactly what is says: start any server, wait for a specific port to be open, and finally run the test. It's very useful in these situations 🙌

To install it, you can use either yarn or npm.

# npm
> npm install start-server-and-test --save-dev

# yarn
> yarn add start-server-and-test --dev

Then inside your package.json, you can add the following commands inside the scripts property:

{
  "scripts": {
    "test": "cypress run",
    "start": "react-scripts start",
    "take-screenshots": "start-server-and-test 3000"
  }
}

To test it, you can simply execute the take-screenshot command and you should be able to see something like:

➜ my-project git:(master) ✗ yarn take-screenshots
yarn run v1.22.5
$ start-server-and-test 3000
1: starting server using command "npm run start"
and when url "['http://localhost:3000']" is responding with HTTP status code 200
running tests using command "npm run test"

...

Now I only need something that will call this command automatically for me and push it into my Repository. This is when GitHub Actions comes into the Game!

DISCLAIMER: you can use any Continuous Integration technology (Travis CI, Circle CI, etc.), but I find the implementation with Actions quite simple and straight forward.

To enable GitHub Actions, you have to create a new Workflow file located inside .github/workflows folder. You can name this file as you want, the only rule to follow is that it has to be a YAML file.

# .github/workflows/update-docs.yml

# Workflow name
name: Update Docs

# Run on every push to master
on:
  push:
    branches: master

jobs:
  # Job name
  update-readme:
    # Environment setup
    runs-on: ubuntu-latest

    # job
    steps:
      - name: Checkout repository
        uses: actions/checkout@v2
      - name: Setup Node
        uses: actions/setup-node@v1
      - name: Install dependencies
        run: yarn install --frozen-lockfile
      - name: Take screenshots
        run: yarn take-screenshots
      - name: Commit screenshots
        uses: stefanzweifel/git-auto-commit-action@v4
        with:
          commit_message: Update Screenshots

Something I like about the Workflows from GitHub Actions is how descriptive they are. It consists of:

Name of the workflow, which serves as an identifier (name).
When the Workflow will be triggered (on).
What it will do once triggered (jobs). Each Job has:
- Unique identifier.
- On which environment it will run (runs-on).
- Set of steps to run scripts or call other Actions (steps).

One of the best features of GitHub Actions is the ability to compose Actions. In the example from above, to commit the changes inside the screenshots I am using the git-auto-commit-action which will look at my changes and then perform a git commit -am "commit_message" into my repository.

Finally, to test if your Workflow is working you have to push your changes into your master branch. Here you can see a normal execution I run inside gatsby-starter-mate.

Taking it to the next level 🚀

In case you have noticed inside the photographer.test.js file, I had to type the name of my routes and the selectors that I wanted to use. But what if, I can also automate this too? 🤔

The first step will be to collect all the routes inside the application. This will change depending on the technology and routing you have picked. But from here you should produce a file called routes.json with the list of all the routes.

Next, you have to store this file inside the folder cypress/fixtures, so then Cypress can have access to it inside the photographer.test.js file:

describe('Dynamic Photographer', () => {
  const routes = require('../fixtures/routes.json');

  beforeEach(() => {
    cy.viewport('macbook-15'); // Desktop viewport
  });

  routes.forEach((route) => {
    it(`${route}`, () => {
      cy.visit('/');
      cy.get('#shared-section-id').screenshot(route);
    });
  });
});

Finally, you have to generate the table based on the routes.json file or from the generated images. In this case, I will do it from the file, but in both cases, the output should be the same.

// generate-docs-readme.js
const fs = require('fs');
const routes = require('./cypress/fixtures/routes.json');

const routeToTable = (route) =>
  `| ${route} | ![${route}](screenshots/photographer.test.js/${route}.png) |`;

const content = [
  '## Pages Screenshots',
  'Dynamic screenshots based on the last version deployed.',
  '| Page | Screenshot |',
  '| --- | :---: |',
  ...routes.map(routeToTable),
];

fs.writeFileSync('photographer.md', content.join('\n'));

This idea can be improved in several ways:

Expose specific CSS selectors depending on the screen, to have more accurate screenshots.
Expose names of screens instead of treating them as routes.
With the combination of the idea from above, your file routes.json can tell Cypress to take more than one picture from a Page.

Last Words

I had a ton of fun integrating these two tools and, I think the result it's quite useful. I hope more people also found it appealing and decided to try it on their own projects to battle against out-dated documentation 😅

In this post, I nearly used all the potential from these two tools. I highly recommend passing by though their official documentation: Cypress Docs and GitHub Actions Docs. They provide much more functionality inside.

Thanks for reading and let's keep building stuff together! 👷‍♂️

References

End-to-end testing in React Native with Detox

Ema Suriano — Fri, 29 May 2020 16:24:00 +0000

Article originally published on the LogRocket Blog.

End-to-end testing is a technique that is widely performed in the web ecosystem with frameworks like Cypress, Puppeteer, or maybe with your own custom implementation.

But when it comes to the mobile world, this practice is not that common, and there are several existing solutions to address. I have a theory that most mobile developers think testing mobile applications is hard and requires a lot of setup and configuration, and therefore they just skip it.

The goal of this article is to explain how to implement the end-to-end testing framework Detox in a React Native application, write a bunch of interaction tests, and, finally, integrate it into your development workflow.

⚠️ Disclaimer ️️⚠️: Given the length of this article, I will only focus on the iOS flow. Nevertheless, I plan to release a second part covering Android, too.

A quick introduction to end-to-end testing 📖

Let’s take a look at the definition of this testing technique:

End-to-end testing is a technique used to test whether the flow of an application right from start to finish is behaving as expected. The purpose of performing end-to-end testing is to identify system dependencies and to ensure that data integrity is maintained between various system components and systems. — Software Testing Dictionary

In contrast to unit testing, end-to-end testing tries to cover as much of your application’s functionality as it can. The more it covers, the more reliable your tests will be. Therefore, it includes all the stages of an application:

Set up environment
Installation of the application (if it’s necessary)
Initialization
Executing routines
Expecting events or behaviors to happen

This is how end-to-end testing looks in the browser using Cypress:

End-to-End Testing with Cypress

Cypress is able to create an instance of Chrome, run a URL, and then start interacting with the webpage by selecting elements (div, button, input) using native selectors (getElementById, getElementByName, getElementByClassName), and then triggering events (click, change, focus).

At any point in the tests, the developer can assert/expect something to happen or to have a specific value. In case all the expectations were true, the result of the test suite will be successful.

End-to-end testing in mobile 🤯

The process of testing mobile applications is actually quite similar to the web. Let’s go thought the previously described steps:

Set up the environment: create an instance of an emulator (Android/iOS device)
Installation: install the application
Initialization: run the application
Executing routines: Depending on the framework this may change, but all of them are using native directives to get the reference of an element (Button, View, TextInput) and then executing actions (press, type, focus)
Expecting events: Using the same functions described before, they can validate values or events that happened

This is how end-to-end testing looks like in mobile using Detox:

End-to-End Testing with Detox

What is Detox, and why should you pick it?

Detox is an end-to-end framework for mobile apps developed by Wix, one of the top contributors inside the React Native community. They also maintain amazing projects such as react-native-navigation and react-native-ui-lib.

What I like about this framework is the great abstraction it provides to select and trigger actions on elements. This is how a normal test looks:

describe('Login flow', () => {
  it('should login successfully', async () => {
    await device.reloadReactNative();
    // getting the reference of an element by ID and expecting to be visible
    await expect(element(by.id('email'))).toBeVisible();

    // Getting the reference and typing
    await element(by.id('email')).typeText('john@example.com');
    await element(by.id('password')).typeText('123456');

    // Getting the reference and executing a tap/press
    await element(by.text('Login')).tap();

    await expect(element(by.text('Welcome'))).toBeVisible();
    await expect(element(by.id('email'))).toNotExist();
  });
});

As you can see, the syntax is quite readable, and by using async/await, you can write synchronous and easy-to-understand tests. Let’s jump into the demo!

Ready, set, code! 🏎

In case you want to skip the explanation and check the code, I’ll give you the link for the repository with the project already bootstrapped and the tests in place.

As the focus of this article is testing and not explaining how to set up React Native, I suggest bootstrapping your project using react-native init, which creates a pretty simple and clean React Native project.

Start by installing the dependency and creating a fresh new project.

~ npm install react-native -g
~ react-native init testReactNativeDetox

               ######                ######
             ###     ####        ####     ###
            ##          ###    ###          ##
            ##             ####             ##
            ##             ####             ##
            ##           ##    ##           ##
            ##         ###      ###         ##
             ##  ########################  ##
          ######    ###            ###    ######
      ###     ##    ##              ##    ##     ###
   ###         ## ###      ####      ### ##         ###
  ##           ####      ########      ####           ##
 ##             ###     ##########     ###             ##
  ##           ####      ########      ####           ##
   ###         ## ###      ####      ### ##         ###
      ###     ##    ##              ##    ##     ###
          ######    ###            ###    ######
             ##  ########################  ##
            ##         ###      ###         ##
            ##           ##    ##           ##
            ##             ####             ##
            ##             ####             ##
            ##          ###    ###          ##
             ###     ####        ####     ###
               ######                ######


                  Welcome to React Native!
                 Learn Once Write Anywhere

✔ Downloading template
✔ Copying template
✔ Processing template
✔ Installing dependencies
✔ Installing CocoaPods dependencies (this may take a few minutes)

  Run instructions for iOS:
    • cd testReactNativeDetox && react-native run-ios
    - or -
    • Open testReactNativeDetox/ios/testReactNativeDetox.xcworkspace in Xcode or run "xed -b ios"
    • Hit the Run button

  Run instructions for Android:
    • Have an Android emulator running (quickest way to get started), or a device connected.
    • cd testReactNativeDetox && react-native run-android

After this step, you can try running the application in the emulator by executing:

~ cd testReactNativeDetox
~ react-native run-ios

Application running

Time to test! 🔧

Before jumping into testing, you need to have the following prerequisites:

Xcode installed
Homebrew installed and updated
Node.js installed (brew update && brew install node)
applesimutils installed (brew tap wix/brew; brew install applesimutils;)
detox-cli installed (npm install -g detox-cli)

Start by adding Detox as a dev dependency for the project.

~ yarn add detox -D

Inside the CLI, they provide a command that can automatically set up the project. You need to run:

~ detox init -r jest

detox[34202] INFO: [init.js] Created a file at path: e2e/config.json
detox[34202] INFO: [init.js] Created a file at path: e2e/init.js
detox[34202] INFO: [init.js] Created a file at path: e2e/firstTest.spec.js
detox[34202] INFO: [init.js] Patching package.json at path: /Users/USERNAME/Git/testReactNativeDetox/package.json
detox[34202] INFO: [init.js] json["detox"]["test-runner"] = "jest";

This will create a new folder called e2e with a basic test and some initial configuration such as init.js, which is the file that tells jest to start the simulator and so on. Let’s modify this initial test to check if the two first sections are visible.

describe('Example', () => {
  beforeEach(async () => {
    await device.reloadReactNative();
  });

  it('should have "Step One" section', async () => {
    await expect(element(by.text('Step One'))).toBeVisible();
  });

  it('should have "See Your Changes" section', async () => {
    await expect(element(by.text('See Your Changes'))).toBeVisible();
  });
});

Next, you need to add a configuration for Detox inside your package.json. Add the following object to the detox key, replacing the name of testReactNativeDetox with the name of your application:

{
  "detox": {
    "test-runner": "jest",
    "configurations": {
      "ios.release": {
        "binaryPath": "./ios/build/Build/Products/Release-iphonesimulator/testReactNativeDetox.app",
        "build": "xcodebuild -workspace ios/testReactNativeDetox.xcworkspace -configuration release -scheme testReactNativeDetox -sdk iphonesimulator -derivedDataPath ios/build",
        "type": "ios.simulator",
        "name": "iPhone X"
      }
    }
  }
}

Once done, try to build the application by running:

~ detox build

In case your build failed with the message clang: error: linker command failed with exit code 1 (use -v to see invocation), please refer to this solution in GitHub issues and try running the command again.

Finally, time to run the test!

~ detox test

 PASS e2e/firstTest.spec.js (7.514s)
  Example
    ✓ should have "Step One" section (260ms)
    ✓ should have "See Your Changes" section (278ms)

Initial Detox setup

Time to make it fancier! 💅

Let’s put those boring and flat sections inside a colorful carousel! Because who doesn’t love them?

In order to save time, I decided to use an existing carousel component built by the community. For this demo, I used react-swipeable-views-native. I’m sure there must be better alternatives out there, but this one was the perfect match for my needs.

Also, in order to generate nice random colors, I used randomcolor.

Install both libraries as dependencies for the project:

~ yarn add react-swipeable-views-native randomcolor

Then I made a few modifications inside App.js — you can find the code here. This is the summary of changes:

Wrap all the sections inside SwipeableViews to enable swiping behavior
Wrap each section inside a custom View called Slide that implements properties like padding and backgroundColor
Add a Button and a TextInput component to the last two slides

And this is the result:

Demo with carousel

Writing Detox tests 🧪

In order to make things easier, let’s add two new scripts into the package.json:

{
  "scripts": {
    "e2e:test": "detox test -c ios.release",
    "e2e:build": "detox build -c ios.release"
  }
}

Now that the application has changed, you need to create a new build of it in order to run tests with the modified version. Execute the following command:

~ yarn e2e:build

This process may take some time. In the meantime, let’s take a quick look at the existing tests:

describe('Example', () => {
  beforeEach(async () => {
    await device.reloadReactNative();
  });

  it('should show "Step One"', async () => {
    await expect(element(by.text('Step One'))).toBeVisible();
  });

  it('should show "See Your Changes"', async () => {
    await expect(element(by.text('See Your Changes'))).toBeVisible(); // THIS TEST WILL FAIL!
  });
});

The second test will definitely fail because the “See Your Changes” section is now in the second slide of the Carousel, which is not visible for the user until they swipe. Therefore, let’s make Detox move to that slide!

describe('Example', () => {
  // previous tests here

  it('should render "See Your Changes" in the second slide', async () => {
    // getting the reference of the slides and make a swipe
    await element(by.id('slides')).swipe('left');
    await expect(element(by.text('See Your Changes'))).toBeVisible(); // no this will pass!
  });
});

At this point, you can execute the end-to-end tests, and they should pass! The command is:

~ yarn e2e:test

 PASS e2e/firstTest.spec.js (7.514s)
  Example
    ✓ should have "Step One" section (260ms)
    ✓ should render "See Your Changes" in the second slide (993ms)

Running a basic test

Let’s add a few more tests to cover the following scenarios:

Test that the carousel allows the user to move back and forth inside the slides.
Move the third slide and interact with the Button
Move the last slice and interact with the TextInput

describe('Example', () => {
  // previous tests here

  it('should enable swiping back and forth', async () => {
    await expect(element(by.text('Step One'))).toBeVisible();
    await element(by.id('slides')).swipe('left');
    await element(by.id('slides')).swipe('right');
    await expect(element(by.text('Step One'))).toBeVisible();
  });

  it('should render "Debug" and have a Button to click in the third slide', async () => {
    await element(by.id('slides')).swipe('left');
    await element(by.id('slides')).swipe('left');
    await expect(element(by.text('Debug'))).toBeVisible();

    await element(by.text('Click here!')).tap();
    await expect(element(by.text('Clicked!'))).toBeVisible();
  });

  it('should render "Learn More" and change text in the fourth slide', async () => {
    await element(by.id('slides')).swipe('left');
    await element(by.id('slides')).swipe('left');
    await element(by.id('slides')).swipe('left');
    await expect(element(by.text('Learn More'))).toBeVisible();

    const docsInput = element(by.id('docsInput'));

    await expect(docsInput).toBeVisible();

    await docsInput.clearText();
    await docsInput.typeText('Maybe later!');

    await expect(docsInput).toHaveText('Maybe later!');
  });
});

Feature fully tested! Let’s run the tests again.

~ yarn e2e:test

 PASS e2e/firstTest.spec.js (22.128s)
  Example
    ✓ should have "Step One" section (268ms)
    ✓ should render "See Your Changes" in the second slide (982ms)
    ✓ should enable swiping back and forth (1861ms)
    ✓ should render "Debug" and have a Button to click in the third slide (2710ms)
    ✓ should render "Learn More" and change text in the fourth slide (9964ms)

Running all tests

Bonus: Running E2E test in CI 🎁

Running tests inside CI is quite important; they basically eliminate the need to do manual testing and prevent shipping bugs to production (in case we have the proper set of tests). For this example, I decided to use TravisCI because it has an amazing integration with GitHub and also provides an unlimited plan for open source projects.

In case you are using GitHub, you can install the Travis Application, create a new plan, and allow it to access your repositories.

After that, you need to create a new file inside your project called .travis.yml, which defines the steps you want to run in CI.

I tweaked the CI configuration inside the Official Documentation of Detox a little bit, and this is the one that works in my case.

language: objective-c
osx\_image: xcode10.2

branches:
  only:
    - master

env:
  global:
    - NODE\_VERSION=stable

install:
  - brew tap wix/brew
  - brew install applesimutils
  - curl -o- [https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh](https://raw.githubusercontent.com/creationix/nvm/v0.33.2/install.sh) | bash
  - export NVM\_DIR="$HOME/.nvm" && [-s "$NVM\_DIR/nvm.sh"] && . "$NVM\_DIR/nvm.sh"
  - nvm install $NODE\_VERSION
  - nvm use $NODE\_VERSION
  - nvm alias default $NODE\_VERSION

  - npm install -g react-native-cli
  - npm install -g detox-cli
  - npm install
  - cd ios; pod install; cd -;

script:
  - npm run e2e:ci

One last thing: add the command e2e:ci into your package.json. This command will build the application (detox build), run the tests (detox test), and close the emulator in order to finish the execution (--cleanup flag).

{
  "scripts": {
    "e2e:test": "detox test -c ios.release",
    "e2e:build": "detox build -c ios.release",
    "e2e:ci": "npm run e2e:build && npm run e2e:test -- --cleanup"
  }
}

Once you pushed all the changes into your master branch, try to open a new pull request. You should see a new pull request checker has been added, which will call Travis, and this one runs the Detox tests.

Travis PR Checker

Here is the link to the full log in Travis for that pull request.

Closing words

In case you were thinking of adding tests to your React Native application, I highly encourage you to go and try Detox! Detox is an amazing end-to-end testing solution for mobile, and after using it for quite some time, these are the pros and cons:

✅ Very well abstracted syntax for matchers and to trigger specific actions
✅ Integration with Jest is just wonderful
✅ Possibility to run tests in CI
❌ Sometimes you might encounter configuration errors, and finding the proper solution may take some time. The best way to address this problem is to go and take a deep look at the GitHub Issues

One more thing before you leave, I decided to start a newsletter so in case you want to hear about what I’m posting please consider following it! No SPAM, no hiring, no application marketing, just tech posts 👌

EmaSuriano Newsletter

References and further reading

Docker for Frontend Developers

Ema Suriano — Fri, 29 May 2020 11:04:44 +0000

Article originally published on the LogRocket blog.

Since Dockers’s release in 2013, the use of containers has been on the rise, and it’s now become a part of the stack in most tech companies out there. Sadly, when it comes to front-end development, this concept is rarely touched.

Therefore, when front-end developers have to interact with containerization, they often struggle a lot. That is exactly what happened to me a few weeks ago when I had to interact with some services in my company that I normally don’t deal with.

The task itself was quite easy, but due to a lack of knowledge of how containerization works, it took almost two full days to complete it. After this experience, I now feel more secure when dealing with containers and CI pipelines, but the whole process was quite painful and long.

The goal of this post is to teach you the core concepts of Docker and how to manipulate containers so you can focus on the tasks you love!

The what and why for Docker 🤔

Let’s take a look at a simple definition of what is Docker:

Docker is a tool that allows developers, sys-admins, etc. to easily deploy their applications in a sandbox (called containers) to run on the host operating system. — Docker Curriculum

The key benefit of using containers is that they package up code and all its dependencies so the application runs quickly and reliably regardless of the computing environment.

This decoupling allows container-based applications to be deployed easily and consistently regardless of where the application will be deployed: a cloud server, an internal company server, or your personal computer.

Docker Architecture

Terminology 📖

In the Docker ecosystem, there are a few key definitions you’ll need to know to understand what the heck they are talking about:

Image: The blueprints of your application, which forms the basis of containers. It is a lightweight, standalone, executable package of software that includes everything needed to run an application, i.e., code, runtime, system tools, system libraries, and settings.
Containers: These are defined by the image and any additional configuration options provided on starting the container, including but not limited to the network connections and storage options.
Docker Daemon: The background service running on the host that manages the building, running, and distribution of Docker containers. The daemon is the process that runs in the OS the clients talk to.
Docker Client: The CLI that allows users to interact with the Docker daemon. It can also be in other forms of clients, too, such as those providing a UI interface.
Docker Hub: A registry of images. You can think of the registry as a directory of all available Docker images. If required, you can host your own Docker registries and pull images from there.

“Hello, World!” Demo 🌎

To fully understand the aforementioned terminologies, let’s set up Docker and run an example.

The first step is installing Docker on your machine. To do that, go to the official Docker page, choose your current OS, and start the download. You might have to create an account.

After installing Docker, open your terminal and execute docker run hello-world. You should see the following message:

➜ ~ docker run hello-world
Unable to find image 'hello-world:latest' locally
latest: Pulling from library/hello-world
1b930d010525: Pull complete
Digest: sha256:6540fc08ee6e6b7b63468dc3317e3303aae178cb8a45ed3123180328bcc1d20f
Status: Downloaded newer image for hello-world:latest

Hello from Docker!
This message shows that your installation appears to be working correctly.

Let’s see what actually happened behind the scenes:

docker is the command that enables you to communicate with the Docker client.
When you run docker run [name-of-image], the Docker daemon will first check if you have a local copy of that image on your computer. Otherwise, it will pull the image from Docker Hub. In this case, the name of the image is hello-world.
Once you have a local copy of the image, the Docker daemon will create a container from it, which will produce the message "Hello from Docker!"
The Docker daemon then streams the output to the Docker client and sends it to your terminal.

Node.js Demo 📦

The “Hello, World!” demo was quick and easy, but the truth is we were not using all Docker’s capabilities. Let’s do something more interesting. Let’s run a Docker container using Node.js.

So, as you might guess, we need to somehow set up a Node environment in Docker. Luckily, the Docker team has created an amazing marketplace where you can search for Docker images inside their public Docker Hub. To look for a Node.js image, you just need to type “node” in the search bar, and you most probably will find this one.

Official Node Images

So the first step is to pull the image from the Docker Hub, as shown below:

➜ ~ docker pull node

Then you need to set up a basic Node app. Create a file called node-test.js, and let’s do a simple HTTP request using JSON Placeholder. The following snippet will fetch a Todo and print the title:

const https = require('https');

https
  .get('https://jsonplaceholder.typicode.com/todos/1', response => {
    let todo = '';

    response.on('data', chunk => {
      todo += chunk;
    });

    response.on('end', () => {
      console.log(`The title is "${JSON.parse(todo).title}"`);
    });
  })
  .on('error', error => {
    console.error('Error: ' + error.message);
  });

I wanted to avoid using external dependencies like node-fetch or axios to keep the focus of the example just on Node and not in the dependencies manager.

Let’s see how to run a single file using the Node image and explain the docker run flags:

➜ ~ docker run -it --rm --name my-running-script -v "$PWD":/usr/src/app -w /usr/src/app node node node-test.js

-it runs the container in the interactive mode, where you can execute several commands inside the container.
--rm automatically removes the container after finishing its execution.
--name [name] provides a name to the process running in the Docker daemon.
-v [local-path: docker-path] mounts a local directory into Docker, which allows exchanging information or access to the file system of the current system. This is one of my favorite features of Docker!
-w [docker-path] sets the working directory (start route). By default, this is /.
node is the name of the image to run. It always comes after all the docker run flags.
node node-test.js are instructions for the container. These always come after the name of the image.

The output of running the previous command should be: The title is "delectus aut autem".

React.js demo ⚛️

Since this post is focused on front-end developers, let’s run a React application in Docker!

Let’s start with a base project. For that, I recommend using the create-react-app CLI, but you can use whatever project you have at hand; the process will be the same.

➜ ~ npx create-react-app react-test
➜ ~ cd react-test
➜ ~ yarn start

You should be able to see the homepage of the create-react-app project. Then, let’s introduce a new concept, the Dockerfile.

In essence, a Dockerfile is a simple text file with instructions on how to build your Docker images. In this file, you’d normally specify the image you want to use, which files will be inside and whether you need to execute some commands before building.

Let’s now create a file inside the root of the react-test project. Name this Dockerfile, and write the following:

# Select the image to use
FROM node

## Install dependencies in the root of the Container
COPY package.json yarn.lock ./
ENV NODE\_PATH=/node\_modules
ENV PATH=$PATH:/node\_modules/.bin
RUN yarn

# Add project files to /app route in Container
ADD . /app

# Set working dir to /app
WORKDIR /app

# expose port 3000
EXPOSE 3000

When working in yarn projects, the recommendation is to remove the node_modules from the /app and move it to root. This is to take advantage of the cache that yarn provides. Therefore, you can freely do rm -rf node_modules inside your React application.

After that, you can build a new image given the above Dockerfile, which will run the commands defined step by step.

➜ ~ docker image build -t react:test .

To check if the Docker image is available, you can run docker image ls.

➜ ~ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
react test b530cde7aba1 50 minutes ago 1.18GB
hello-world latest fce289e99eb9 7 months ago 1.84kB

Now it’s time to run the container by using the command you used in the previous examples: docker run.

➜ ~ docker run -it -p 3000:3000 react:test /bin/bash

Be aware of the -it flag, which, after you run the command, will give you a prompt inside the container. Here, you can run the same commands as in your local environment, e.g., yarn start or yarn build.

To quit the container, just type exit, but remember that the changes you make in the container won’t remain when you restart it. In case you want to keep the changes to the container in your file system, you can use the -v flag and mount the current directory into /app.

➜ ~ docker run -it -p 3000:3000 -v $(pwd):/app react:test /bin/bash

root@55825a2fb9f1:/app# yarn build

After the command is finished, you can check that you now have a /build folder inside your local project.

Conclusion 👋

This has been an amazing journey into the fundamentals of how Docker works. For more advanced concepts, or to cement your understanding of the discussed concepts in this article, I advise you to check out the references linked below.

EmaSuriano Newsletter

References 🤓

Building a collaborative Calendar with Google and Gatsby

Ema Suriano — Wed, 27 May 2020 12:44:15 +0000

About one week ago a friend of mine came to me for help, he wanted to create an online calendar for cultural events around the city. The idea was to create an application with a calendar showing all the upcoming events with the possibility that any person can add or edit new events.

So in summary, the application needs to have the following features:

Display a calendar with events and information about them.
Read the events from somewhere.
Provide a way to add new events.

Breaking down the Challenge 👷‍♂️

If we translate these requirements to technical points, I listed the following requirements:

Storage: Have a database to store the events
Backend: Provide an API to read/write from/to the database.
Frontend: A web app with two pages, one to add the events and another with a calendar to display the events.

Seems like a lot of work right? But what if I only focus on the frontend and delegate the backend and storage? Now it doesn’t seem that long!

Most probably you heard about Google Forms and Google Sheets? Let me tell you that they work amazing as Backend and Storage 😄 This is how it works:

Google Forms provides the page to add and edit events to the calendar. The only thing you need to do is ask the proper questions.
Google Forms has a hidden feature to save your results inside a Google Spreadsheet, so each answer will be mapped into a row inside of a table.
Google has made an excellent job by providing APIs for all their products (you just have to enable it), therefore now we can read the rows from the spreadsheets from our application.
The missing part is the Frontend, I used Gatsby to create the website with the calendar because of its great Source Plugin system, allowing me to access the events information very easily.

I made a simple diagram to show the interaction between the different parts. Hope it’s clear now 😅

Setting the Form 📝

Pretty straight forward, create a new Google Form asking all the questions regarding your events. Most probably you want to ask for the name, when, where, etc.

Here is the link to the form I created in case you want to check how I structure the questions.

Saving answers inside a Spreadsheet 💾

Inside the Edit Mode of the Form, select the Responses and click on the green Spreadsheet icon that says “View responses in Sheets”.

This will automatically create a Spreadsheet for you linking each answer of the form to a column. And of course, every time someone adds a new event the spreadsheet will be updated too ✨

Enabling Google API to read from the spreadsheet

This step could be tricky the first time you do it, but like the others is super simple too. Navigate to your Google API Dashboard and open the Credentials located in the left panel. There click on “Create credentials” and select “Service account key”.

You’ll be redirected to a new page to create the service account key, make sure the option of “App Engine default service account” is selected and the key type marked as JSON. Click on Create and you will receive a JSON file with a lot of fields, for the moment store it somewhere on your computer because we are going to use more than once.

Inside the JSON object, look for the key client_email and copy the value of it which should be a valid email address. Open the Spreadsheet generated by Google Forms, click on the Share button located at the top right corner, and add the mail. This is how you enable to read the document from an external application 🎉

Displaying the events within a calendar 🗓

Once we stored all the events inside the Spreadsheet and have a way to retrieve them, it’s time to add a proper UI which means a beautiful Calendar!

I created a Gatsby starter with a 1-minute setup that connects the application with your spreadsheet and renders all the events inside a calendar. You can find the repository here.

In order to bootstrap the project, I recommend using the new command from Gatbsy, which will create a new folder, copy all the code from the repository and install all the dependencies.

> gatsby new event-calendar https://github.com/EmaSuriano/gatsby-starter-event-calendar

Remember the JSON file you downloaded before? Now it’s time to use it! Create a new called .env in the root of the project with the following information from your config file.

PRIVATE_KEY= // value of private_key
PRIVATE_KEY_ID= // value of private_key_id
PROJECT_ID= // value of project_id

The project is going to read all these credentials and grant access to access the information inside your Spreadsheet. This file is excluded inside the .gitignore, therefore it would never be published.

After this open gatsby-config.js and look for options of gatsby-source-google-sheets, and change the values of:

spreadsheetId: you can find this value at the ending of the Spreadsheet URL. https://docs.google.com/spreadsheets/d/[spreadsheetId]
worksheetTitle: this is the name of the sheet from which you want to read the information.

Finally, you need to map your columns of the spreadsheet to the data structure that the calendar is expecting. I highly suggest running your queries inside http://localhost:8000/___graphql, which is the GraphQL playground where you can write your queries and get the result immediately.

In my case, I have to write the following query:

    query eventsQuery {
        allGoogleSheetEventsRow {
          edges {
            node {
              id
              eventName: whatisthename
              date: when
              place: where
              eventLink: linktotheevent
            }
          }
        }
      }

Now you are ready to run the project, fetching all the events inside your Spreadsheet and display them inside beautiful Calendars!️ Execute the following command inside your terminal:

> yarn start

This command will trigger gatsby develop that will run all the Static Queries inside the project and host your application inside http://localhost:8000/. If you open it inside your browser you should see this Home page:

And if you scroll just a little bit or click on See all the events button, you should be able to see all the Calendars filled with the information of your events 🎉

In case you want to know more about Event Calendar Starter, I recommend you to check the documentation that includes sections not present in this article, like App Configuration, Theming, Project Structure, Deployment.

Last words 👋

I hope you find this article interesting and it has encouraged you to build your own events page for your own community or friends.