DEV Community: Halo lab

How to Run a Python script from Node.js

Chuiko Liliia — Tue, 14 Mar 2023 11:14:00 +0000

Quite often we hear disputes about what is better to use Node.js or Python.
But what if one day, you come up with a great idea to combine those, for instance to use some Python libraries in your application, but you just do not have any idea how to integrate it with your Node.js application. Of course you can always build API on top of Python backend(Flack, etc), but in that case you need to build, host and manage one more application, when you just need to run a single Python script. That's why I want to give you step by step instructions on how to achieve this.

To begin with, let's briefly define what are both languages.

Node.js is a server-side platform developed on Google Chrome’s Javascript Engine. Node Js allows to develop data-intensive real-time web applications. It is written on JavaScript and allows to run applications on various operating systems such as Windows, Linux, and Mac OS.

Python is an object-oriented programming language used to create dynamic web applications. The syntax of Python and dynamic typing make it an ideal programming language for scripting.

Why you might need to run Python scripts from Node.js

Node js can't run heavy multitask as it works on a single thread basis, making it difficult for Node Js to execute CPU-bound tasks. Whenever any request is received, the system will complete the request once and only accept the other request. This process slows down the system and ends up in a massive delay.
Python is much more suitable for that back-end applications, that use numerical computations, big data solutions, and machine learning.
Python has one of the largest ecosystems out of any programming community and a lot of open-source libraries. For instance some of built-in AI libraries (Pylearn2, Tensorflow for neural networks, and Scikit-learn for data analysis)

Сall your Python script from Node.js using Child Process module

You will find several libraries that can help you with this issue. As for me, I prefer to use built in child_process. The node:child_process module provides the ability to spawn subprocesses. This capability is primarily provided by the child_process.spawn() function.

First, we need to create a server.js file as the server for our app. Where we suppose that X and y are large arrays.


const express = require('express');
const { spawnSync } = require('child_process');
const { readFile } = require('fs/promises');
const { appendFile } = require('fs/promises');
const { join } = require('path');
const app = express();

app.get("/", async (req, res, next)=>{
   const X = [1,2, 5] // large array
   const y = [[1,2], [2,3], [1,2]] // large array

   await appendFile(
            join(`scripts/args.json`),
            JSON.stringify({ X, y }),
            {
                encoding: 'utf-8',
                flag: 'w',
            },
        );
   const pythonProcess = await spawnSync('python3', [
     '/usr/src/app/scripts/python-script.py',
     'first_function',
     '/usr/src/app/scripts/args.json',
     '/usr/src/app/scripts/results.json'
   ]);
  const result = pythonProcess.stdout?.toString()?.trim();
  const error = pythonProcess.stderr?.toString()?.trim();

   const status = result === 'OK';
  if (status) {
    const buffer = await readFile('/usr/src/app/scripts/results.json');
    const resultParsed = JSON.parse(buffer?.toString());
    res.send(resultParsed.toString())
  } else {
    console.log(error)
    res.send(JSON.stringify({ status: 500, message: 'Server error' }))
  }
});

//Creates the server on default port 8000 and can be accessed through localhost:8000
const port=8000;
app.listen(port, () => console.log(`Server is listening on PORT ${port}`));

The first parameter, that was passed to the spawnSync is command<string>(the command to run, in this case is python3 - the version of python that is required to run your libraries), the second is args <string[]>(list of string arguments). You can also pass the third parameter options, the more detailed information can be found by the link.
I propose to consider in more detail the list I am passing as the second argument

'/usr/src/app/scripts/python-script.py' - is the path of the script I need to execute
'first_function' - is the name of the function I want to run inside that file
'/usr/src/app/scripts/args.json' - is the path to the file with arguments to this function. Actually you can directly pass those arguments using JSON.stringify() function in Node.js, but as for me it's better to write those arguments down into the file and then read that from Python script. One of the reasons is that you might need to pass as arguments large arrays of data and this will cause errors.
'/usr/src/app/scripts/results.json' - is the path, where I want my Python script to write results, after it is finished, so I can read that from node.js.

Passed arguments can be found in sys.argv array in Python script. The sys.argv list is a list that contains command line arguments in a python program. Whenever we run a python program from a command line interface, we can pass different arguments to the program. The program stores all the arguments and the file name of the python file in the sys.argv list. This can be seen in more detail by writing a simple example script

import json
import sys

def first_function():
    json_obj = open(sys.argv[2])
    data = json.load(json_obj)
    calculatedResults = [1,2,4,3] # it is just example of data to return, in fact you will calculate it bellow
    X = data["X"]
    y = data["y"]
    # do some your calculations based on X and y and put the result to the calculatedResults
    # print(make_pipeline)
    json_object_result = json.dumps(calculatedResults, indent=4)

    with open(sys.argv[3], "w") as outfile:
        outfile.write(json_object_result)
    print("OK")


if sys.argv[1] == 'first_function':
    first_function()

sys.stdout.flush()

In two words, in Node.js script we write down to the file all required arguments, run spawnSync passing list of arguments and after Python script reads passed arguments from the file, makes all calculations and writes down to the file all results. At the moments all this results can be read in Node.js from file.

Run Python script, that uses specific Python libraries

One of the main goals is to have ability to use built-in AI or other libraries. It's always not a good idea to install dependencies and libraries globally in a system, it can lead to version conflicts, deployment difficulties, and problems with multiple developers working on a project at the same time, for example. That's why the perfect way is to use Docker. You can build the image yourself, but the easiest solution is to use an existing one.

FROM nikolaik/python-nodejs:latest as development
WORKDIR /usr/src/app
RUN pip install scikit-learn numpy pandas simplejson
COPY ./package.json ./yarn.lock ./
RUN yarn
RUN yarn install
COPY ./ ./
RUN yarn build
ENV PORT 4000
EXPOSE $PORT


# Production mode
FROM nikolaik/python-nodejs:latest as production
RUN pip install scikit-learn numpy pandas simplejson
ARG NODE_ENV=production
ENV NODE_ENV=${NODE_ENV}
WORKDIR /usr/src/app
COPY --from=development /usr/src/app ./
ENV PORT 4000
EXPOSE $PORT
CMD ["node", "dist/main"]

For now we can use scikit-learn, numpy, pandas and simplejson libraries on Python script, as they are available in the created docker container.

By following the steps above, you can create a new process and execute a Python script from your Node.js application. This can be useful in cases where you need to integrate Python code into your Node.js application or when you need to perform certain tasks that are better suited for Python. Described simple app can be downloaded from the Github, there you can also find instructions how to run it.

Happy coding and wish you to have a nice day!

Implementing a grid system using a simple hand-written single-pass compiler in Sass

Kapelianovych Yevhen — Sun, 16 Jan 2022 09:12:46 +0000

What’s the first thing that pops up in your mind when you think about Sass? Yes, it is the powerful CSS preprocessor that is widely used now. You may also say that it has convenient features for encapsulating repeated parts of the CSS or some complex cases. You might mention nesting, placeholders, etc. But Sass is more powerful than that because it has SassScript.

So, what is this?

SassScript is a set of extensions that can be included in the Sass documents to compute variables from property values and uses properties of variables, arithmetic, and other functions. That gives us the ability to organize a code into functions, perform conditional operations, and many more. Actually, these two abilities are enough to build a whole program.

So, let’s start.

Typical parser implementations theory defines using a character reader, which is used by a lexer that produces tokens, and the last one is the parser itself which creates the AST.

We will build a less capable compiler because Sass has some limitations. Sass cannot work with a filesystem in any way. Because of that, we cannot store custom syntax in separate files. So we should write it inside Sass files. Also, it is inconvenient to process a character's stream because Sass does not provide the most valuable methods for working with strings (and it does not have streams at all). The same goes for the lexer. But we can simulate the latter if we assume that all keywords are already tokens. Sass has a list structure where a space can be a separator between items. And there is a sass:list module that provides some functions to work with lists. We are going to use that ability to simulate the lexer skipping the character sequence stream at all. All the code we are going to write as an argument to a value function. And despite the length of the words, it will still be the list!

value(sequence of the tokens);

Let me describe the idea.

We plan to create a simple responsive grid system. For that, we will assume that there is a table with 64 columns (there may be more or less, it depends) which covers a whole page. Each cell of the table is a square figure. It gives us responsiveness because, on mobile phones, cells will have a smaller size than on laptops. Based on that, we receive a dynamic value (one side of the cell) that we use to express all dimensions in our grid.

You may want to use rem or vw for that purpose. And that’s okay. It’s up to you how to define a unit value.
With that, we’ll be able to say, for example, that “this block should have a width of 5 columns“, and blocks will resize according to unit value changes.

Okay, let’s jump into the code now.

The simplest thing is to calculate the unit value. We know that the grid will have 64 columns. So, the width of one cell can be defined like this:

@use 'sass:math';

:root {
  -—unit: math.div(100vw, 64);
}

Later, I won’t write imports of standard Sass’s modules for brevity. Assume that they are present already.

We took a benefit from CSS variables here and made it global. That will allow us to access this value everywhere in the runtime. We can place it into the mixin named init to allow the developer to override the columns count.

@mixin init($columns: 64) {
  :root {
    —-unit: math.div(100vw, #{$columns});
  }
}

Also, you can define lower and upper bounds for the unit value by using clamp CSS function, but we won’t do that to simplify the code examples.

Move on to the syntax. We're expecting our grid will work for the following use-cases:

value(8); // Just the columns count
value(from 6 to 21); // A distance from 6th column to the 21th column
value(8 min 20px max 70px); // The columns count with the lower and upper bounds of the result value
value(from 6 to 21 min 20px max 70px); // The same but with a range between columns

We have defined the two blocks of the expression: dimension and bounds. The latter is optional. Let’s dive deep into the dimension block a little bit.

The simplest case is when it equals the number - the columns count (see above). There is no more to discuss. The next variant is more difficult. It includes two mandatory keywords: from and to, and two interchangeable: start and end. The last two keywords equal to the 0 and 64 (or whatever number you decided to use) accordingly: from start to end. That expression is equal to from 0 to 64 or 100vw.
The next block has only one variant: min <number with unit> max <number with unit>. As numbers, you can use whatever value you want: pixels, rems, percentages, and so on.

That’s all. Now, when we have all rules defined, we shall go to the most exciting part - the code.

I already said that we would operate on the list of tokens, and for that purpose, we will use the list structure. Unfortunately, there are not enough methods to work with lists in SassScript. So, let’s implement missing ones.
Firstly, we need the isEmpty function to determine whether a list has tokens or not.

@function isEmpty($list) {
  @return list.length($list) == 0;
}

That function is pretty straightforward and does not require additional explanation.

Along with that, we should be able to remove processed tokens from the list. For that, we are going to implement a generic slice function.

@function slice($list, $from, $to: list.length($list) + 1) {
  $_separator: list.separator($list);
  $_copy: ();

  @for $index from $from to $to {
    $_copy: list.append($_copy, list.nth($list, $index), $_separator);
  }

  @return $_copy;
}

Fun fact: all functions that are implemented in SassScript are pure.

It reminds the code from JavaScript with the difference that the first index of the list is 1, unlike 0 in JavaScript. Also, we should preserve a separator of an original list to the sliced, and that’s all.

With that, we can start writing our main value function.

@function value($tokens) {
  $_copy: $tokens;
  $_numbers: (0, 0);
  $_borders: ();

  @if isNumber(list.nth($_copy, 1)) {
    $_numbers: (0, number($_copy));
    $_copy: slice($_copy, 2);
  } @else {
    $_numbers: range($_copy);
    $_copy: slice($_copy, 5);
  }

  @if not isEmpty($_copy) {
    $_borders: borders.borders($_copy);
  }

  @if not isEmpty($_borders) {
    @return clamp(#{list.nth($_borders, 1)}, calc(var(--unit) * #{list.nth($_numbers, 2) - list.nth($_numbers, 1)}), #{list.nth($_borders, 2)});
  }

  @return calc(var(--unit) * #{list.nth($_numbers, 2) - list.nth($_numbers, 1)});
}

Let’s stop here. That is a huge function. Because the syntax is tiny and unambiguous, we can rely on the order of tokens. We know that first is the dimension block, and the second is the borders block. The first if/else block has the code for parsing tokens of the dimension block. Also, you saw $_numbers and $_borders variables. We could define them later, but it is better to define them with a default value to avoid possible errors while using it.

You may notice that there are unknown functions: isNumber, number and range. Let’s define them.

@function isNumber($value) {
  @return meta.type-of($value) == "number";
}

@function number($list) {
  $_value: list.nth($list, 1);

  @if not isNumber($_value) {
    @error "#{$_value} is not a number!";
  }

  @return $_value;
}

isNumber, as it states from the name, checks if the value has a number type. number function takes a first token from the list and makes sure that it is a valid number.

The range function is a bit complex.

@function range($list) {
  $_copy: skip(from, $list);
  $leftNumber: toNumber(list.nth($_copy, 1));
  $_copy: skip(to, slice($_copy, 2));
  $rightNumber: toNumber(list.nth($_copy, 1));

  @if $leftNumber > $rightNumber {
    @error 'From number cannot be greater than To';
  }

  @return ($leftNumber $rightNumber + 1);
}

Simply put, this function reads tokens and checks if they are valid. There are yet other functions that we should provide: skip and toNumber.

@function toNumber($value) {
  @if $value == keywords.$start {
    @return 0;
  }

  @if $value == end {
    @return unit.$columns;
  }

  @if isNumber($value) {
    @return $value;
  }

  @error '#{$value} is not a number or "#{keywords.$start}"/"#{keywords.$end}" keywords.';
}

@function skip($word, $list) {
  @if $word != list.nth($list, 1) {
    @error 'Word #{$word} does not match the #{$list} sequence.';
  }

  @return slice($list, 2);
}

toNumber function is needed to convert start and end keywords to corresponding numbers and check whether the starting column’s value is lower than the ending column’s value. skip is just a convenient method to skip the first token in the list.

range should return starting and ending column numbers to the value to calculate an actual value.
Okay, the dimension block is ready. The borders block remains only.

@function borders($tokens) {
  $_copy: $tokens;
  $_min: 0;
  $_max: $_min;

  $_copy: skip(min, $_copy);

  $_min: list.nth($_copy, 1);

  $_copy: slice($_copy, 2);

  $_copy: skip(max, $_copy);

  $_max: list.nth($_copy, 1);

  @return ($_min, $_max);
}

Here the logic is the same as in the range function. At that point, we have a complete tiny compiler though it looks somewhat different than typical implementations. But the primary goal of this article is to show that Sass is much more powerful than it seems, and even though CSS gets many features that Sass has, the latter is still in demand.

Now, we can use our super compiler to write dimensions in the 64 columns' system.
The example of the code:

@use 'grid' as *;

.some-element {
    margin-left: value(1);
    width: value(from 1 to 8);
    height: value(3);
    background-color: tomato;
}

.some-other-element {
    width: value(from 10 to 16);
    margin-left: value(1);
    height: value(3);
    background-color: tomato;
}

That gives us simpler management of an element’s position. Thus it can be used as a simple replacement for the common grid systems. Also, it can help you to be closer to design because designers use similar grids to combine elements into the whole page.

Personally, it reminds me of a mix of Bootstrap and Susy:

The further development of that idea you can see here. There is an example page that shows simple elements positioning inside the grid. We implemented that idea on our site, that's why we can say it works and works well.

Thank you for reading, and have fun!

Part 2. Quick guide to make your Azure applications work.

Chuiko Liliia — Wed, 05 Jan 2022 12:58:11 +0000

Introduction

In the previous article, we have already created an Azure App service instance. And now you are able to see the Microsoft Azure default page by the link. This time we are going to migrate Strapi app, Next.js app and Mongo DB to Azure.

Preparing Azure App Services for the Migration process

Your project probably has specific environment variables. They should be added to Azure.

Go to the Azure App service instance, find the Configuration in the Settings tab, select Application Settings section.
Here are two kinds of it. Application Settings and Connection Strings. There is only one case for using strings instead of settings, more detailed information can be found by the link. Use Application settings and click on +New application setting.
Put the name and value of the variable and select if it is a deployment slot setting. With this setting in place, database connection strings and other settings are not swapped when the slots are swapped. So the staging slot will always point to the staging database, and the production slot will always point to the production database.

Keep in mind that the values you set in App Service override the values in Web.config and others.

Later I will talk about different deployment methods. But no matter which one you choose, you need to know that deploying new code directly to the production application is a bad practice. For this purpose, every App service instance has staging slots. Deploying your application to a non-production slot has the following benefits:

You can validate app changes in a staging deployment slot before swapping it with the production slot.
Deploying an app to a slot first and swapping it into production guarantees that all instances of the slot are warmed up before being swapped into production. This eliminates downtime when you deploy your app. The traffic redirection is seamless, and no requests are dropped because of swap operations. You can automate this entire workflow by configuring autoswap when pre-swap validation isn't needed.
After a swap, the slot with the previously staged app now has the previous production app. If the changes swapped into the production slot aren't as you expect, you can perform the same swap immediately to get your "last known good site" back.

So the best idea is to create a preview slot that will not affect the production. Creating the slot for an app:

Go to Azure App service instance, find the deployment slots in the Deployment tab, click on Add slot
Put the slot name. It will take your main slot root domain name + name that you will write.
After that, you can select if you need to clone all settings from the main slot or not. If you clone settings, it will make the full copy of the existing slot with the same environment variables. So if you want to use another database or change other settings, you shouldn't clone settings or change them manually after the new slot is ready.

In a few minutes, the preview slot will be available by its own url and can be found in the Deployment Slots tab of the Azure App service instance. And we can proceed to deploy.

Choosing the deployment way

There are a few options to move your project to the instance. Most popular are:

You can deploy to Azure App Service as part of your continuous deployment (CD) workflows via GitHub, Azure Repos
Try Azure Repos with Azure App, make sure you have already created an Azure DevOps Organization to store your project files.
Or use the Azure App Service extension for VS Code to create quickly, manage, and deploy your websites. Just press the install button.

The simplest way for me is deploying via VScode. You will be able to create and deploy resources with it. But this method has a list of disadvantages for my purposes and for. Probably this deployment way will be described in my other article, and for now, deploying using workflows via GitHub was chosen for my project.

Steps to deploy my Strapi back-end application (or any other node.js app) to Azure via GitHub:

Certainly, to achieve this target, your app should be connected to the remote git repository.
Go to your project and create the .github folder at the root of it.
In the .github folder create workflows folder.
In workflows folder create the file with the following name scheme branch_name_app_name(slot_name).yml
For example: App-name: ‘production-app’ Branch from which you want deploy: master Slot: ‘preview’ Final name of the file: master_production-app(preview).yml
Now configure the created .yml file with the following settings. Here is an example that uses the above-declared names:

  push:
    branches:
      - master
  pull_request:
    branches:
      - master

env:
  AZURE_WEBAPP_NAME: production-app  # set this to your application's name
  AZURE_WEBAPP_PACKAGE_PATH: "."      # set this to the path to your web app project, defaults to the repository root
  NODE_VERSION: '14.x'                # set this to the node version to use
  API_URL: "https://production-app.azurewebsites.net" # main slot url

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
    - uses: actions/checkout@master
    - name: Set Node.js version
      uses: actions/setup-node@v1
      with:
        node-version: '14.x'

    # install dependencies, build, and test
    - name: npm install, build, and test  #describes which commands need to be run
      run: |
        npm install
        npm run build --if-present
        npm run test --if-present

    - uses: azure/webapps-deploy@v2
      with:
        app-name: 'production-app’'
        slot-name: 'preview'
        publish-profile: ${{ secrets.AzureAppService_PublishProfile_PRODUCTION }}
        package: .

In the file, all seems to be clear. The only exception is some odd variable used on the penultimate line of code. This setting is required to secure your deployments. Reproduce the following steps to get the value :

a. Go to your preview slot instance and select Get Publish Profile. The file will be downloaded. Open it with the notepad or use your imagination.

b. Copy all content of the opened file.
c. Go to your repository on GitHub -> Settings -> Secrets and press New Repository Secret button.
d. Add name AZUREAPPSERVICE_PUBLISHPROFILE_PRODUCTION in value input, paste the copied content from the file and save changes. In two words, this variable connects your repo with the application instance on Azure.

So, when this thousandth step is complete, you can finally push your code to GitHub by directly pushing or creating a pull request to master-branch. Now it is time to take a cup of tea and just follow the action with full deployment logs on GitHub in Actions tab. It may take some time…

What happens now? First of all, GitHub sets the node.js version, installs all dependencies from package.json, and runs the build command. If there are no errors during building, it zips all the files without the node-modules folder and passes them to Azure. Azure does the same steps: extracts files, installs node modules, creates the production build, then removes the previous project folder and instead puts the new one. After it starts the new Docker container and it is warmed up, it runs the npm start command and starts the app.

Don’t be nervous while waiting. The action can take up to 10 minutes or even more for the first time deploying. When the action is completed, you can visit the link of the preview slot. If you are lucky, you will see your app working. But don’t forget the main Microsoft concept: Microsoft — Just restart it. Check if everything works, and then proceed to swap the app to the production slot. To do this, go to the production Azure App service to Deployment slots, find and press the swap button. Select which slot to swap and to which, confirm the action and wait about 5 minutes. When the swap process is ready, you will get the message. And now all your changes are available on the production slot, and the preview slot link should lead you to the example Microsoft Page.

Steps to deploy my Next.js application to Azure via GitHub.

Actually, the steps to deploy next.js are almost the same as for Node.js application. There are some small distinctions. Next.js project doesn’t have the obvious server.js file that is required to deploy it to Azure App Service. There is also another way: deploy to Static Web App Azure service. But it also can’t be used in a simple way because Next.js has the server-side part, which leads to difficulties with deploying it as the static app. So here is what we should do to deploy Next.js app to Azure App Service, besides what we have already done for the previous app:

In the root project folder create server.js file with the next content and install all missing dependencies:

const { createServer } = require("http");
const next = require("next");

const port = process.env.PORT || 3000;
const app = next({ dev: false });
const handle = app.getRequestHandler();

app.prepare().then(() => {
  createServer((req, res) => {
    handle(req, res); 
  }).listen(port, (err) => {
    if (err) throw err;
    console.log(`> Ready on http://localhost:${port}`);
  });
});

Change your start script in the package.json file to “node server”.
In the projects root create web.config file with the code:

<?xml version="1.0" encoding="utf-8"?>
<!--
     This configuration file is required if iisnode is used to run node processes behind
     IIS or IIS Express.  For more information, visit:
     https://github.com/tjanczuk/iisnode/blob/master/src/samples/configuration/web.config
-->

<configuration>
  <system.webServer>
    <!-- Visit http://blogs.msdn.com/b/windowsazure/archive/2013/11/14/introduction-to-websockets-on-windows-azure-web-sites.aspx for more information on WebSocket support -->
    <webSocket enabled="false" />
    <handlers>
      <!-- Indicates that the server.js file is a node.js site to be handled by the iisnode module -->
      <add name="iisnode" path="server.js" verb="*" modules="iisnode"/>
    </handlers>
    <rewrite>
      <rules>
        <!-- Do not interfere with requests for node-inspector debugging -->
        <rule name="NodeInspector" patternSyntax="ECMAScript" stopProcessing="true">
          <match url="^server.js\/debug[\/]?" />
        </rule>

        <!-- First we consider whether the incoming URL matches a physical file in the /public folder -->
        <rule name="StaticContent">
          <action type="Rewrite" url="public{REQUEST_URI}"/>
        </rule>

        <!-- All other URLs are mapped to the node.js site entry point -->
        <rule name="DynamicContent">
          <conditions>
            <add input="{REQUEST_FILENAME}" matchType="IsFile" negate="True"/>
          </conditions>
          <action type="Rewrite" url="server.js"/>
        </rule>
      </rules>
    </rewrite>

    <!-- 'bin' directory has no special meaning in node.js and apps can be placed in it -->
    <security>
      <requestFiltering>
        <hiddenSegments>
          <remove segment="bin"/>
        </hiddenSegments>
      </requestFiltering>
    </security>

    <!-- Make sure error responses are left untouched -->
    <httpErrors existingResponse="PassThrough" />

    <!--
      You can control how Node is hosted within IIS using the following options:
        * watchedFiles: semi-colon separated list of files that will be watched for changes to restart the server
        * node_env: will be propagated to node as NODE_ENV environment variable
        * debuggingEnabled - controls whether the built-in debugger is enabled
      See https://github.com/tjanczuk/iisnode/blob/master/src/samples/configuration/web.config for a full list of options
    -->
    <iisnode watchedFiles="web.config;*.js"/>
  </system.webServer>
</configuration>

Reproduce all steps for deploying node.js app.

Congrats! Deploying both front-end and back-end applications is finally finished. The last thing we need to migrate is the Mongo database.

The first and main thing you should know before starting the migration database to Azure Cosmos DB for Mongo – don’t ever do this!!! Strapi and Azure Cosmos DB for Mongo are almost incompatible. Actually, I still don’t understand what Azure Cosmos for Mongo is compatible with.

It doesn’t support a lot of operators and expressions that Strapi uses. For example: $let, $where, $meta and others. I have made a migration and faced indexing problems and a lot of unsupported operators. After a huge amount of work was done and after most of the problems were fixed, it finally started to work together. But when I updated the Strapi to the new version, I got one thousand additional problems. And finally, official representatives of the company Strapi gave me the answer:

Technically CosmosDB is not entirely MongoDB (even with the API implemented) It's not on the list of supported databases, and may not fully work with the ORM being used (mongoose).

Moreover, it has a complicated way to take backups. The system takes snapshots, but you can't get them immediately. Firstly you need to create a support request for the Microsoft Support team. It can be done manually in 2 minutes in Mongo Atlas. Just select the required snapshot and click the restore button. Looks much simpler, doesn't it?

Don't waste time fixing problems, there is an easy way to avoid them. If your target is just to speed up the application, use Mongo Atlas for hosting the database and select the Azure cloud provider and the region closest to your customers. In case you need more options for regions, backup settings and some other benefits, learn about dedicated cluster tier features.

If Mongo Atlas cluster is hosted on Azure servers, most of the requests will not go over the public network, they will use the Azure backbone network instead. Which is more secure and the fastest way to deliver the data.

In the end, we have two applications moved to Azure and Mongo DB hosted on the Mongo Atlas. It is the optimal solution for my purposes. But be ready to read more about Azure, believe me, you will definitely need it.

From zero to one. How to Deploy your MongoDB, Node, Strapi, NextJS app to Microsoft Azure.

Chuiko Liliia — Wed, 29 Dec 2021 15:18:26 +0000

Part 1. Introduction and creating all required for migration resources.

Introduction

Hi to everyone! This article will be helpful if you have little or no knowledge of cloud technologies.

There are three main cloud providers: Google Cloud, AWS and Azure. Let’s imagine that your task is to move the application to Azure.

Before continuing, note that the best way to familiarize with basic Azure cloud features, to gain a foundational knowledge of cloud services, and the way those services are provided with Microsoft Azure is to learn and pass Exam AZ-900: Microsoft Azure Fundamentals. The exam is for candidates who are just beginning to work with cloud-based solutions and services or are new to Azure. Azure Fundamentals can be used to prepare for other Azure role-based or specialty certifications, but it is not a prerequisite for any of them. All Microsoft Azure learning has two options: online, which is always free, and instructor-led, which is paid. Passing the exam and getting the certification is always paid. The price depends on the country. One more tip: learn it before migration because learning during the process is a bad idea. There is much to say about passing Microsoft exams, but I suggest we move on to the next step and explore the project structure.

Project structure and reasons for migration to Azure

So, let’s do a deep dive into the project structure before moving on.

Front-end: NEXT.js app, hosted on Vercel.
Back end: Strapi CMS + custom node scripts, hosted on Heroku
DB: Mongo db hosted on Mongo Atlas

The common problems with current providers include:

No simple ways to spread application to multiply regions and elastically scale the capacity based on demand.
No static inbound and outbound IP addresses (only via proxy), which I needed to secure back-end and DB connections.
No obvious ways to give separate accesses to resources, based on roles.

First steps to start to work with Azure, regardless wich computing services you choose to host applications

Firstly go to Microsoft Azure Portal and create the account. After you complete the registration, you will have a single tenant ID associated with your account, which will not change unless you ask Microsoft to delete your account. Tenant is associated with a single identity (person, company, or organization) and can own one or several subscriptions. At this step you will only have one subscription. An Azure subscription is a logical container used to provision resources in Azure. It holds the details of all your resources like virtual machines (VMs), databases, and more. When you create an Azure resource like a VM, you identify the subscription it belongs to. A subscription is linked to a payment setup and each subscription will result in a separate bill and can use different payment methods. Resources from one subscription are isolated from resources in other subscriptions.

During the registration in Azure you may choose to get 200$ dollars on balance of the default subscription that you can spend on Azure services. It is a very useful option for beginners in cloud technologies to test different options.

Now you are signed in, have a tenant and one subscription inside it. The next step is to decide which Azure services to use for your apps hosting.

Choose the appropriate Azure services

Azure offers three main cloud computing platform services: SaaS (Software as a Service), IaaS (Infrastructure as a Service), PaaS (Platform as a Service). The main differences and maintaining levels are displayed on the diagram below:

If you don’t want to be responsible for the operating system, data, runtime and middleware, but still need to manage and configure the application, the PaaS cloud computing service is a good choice.

So, PaaS — it is where Azure provider hosts the hardware and software on its own infrastructure and delivers this platform to the user as an integrated solution, solution stack, or service through an internet connection, while the developer is allowed to develop, run, and manage his own apps without having to build and maintain the infrastructure or platform usually associated with the process.

When we have decided which cloud computing service to use, let's proceed to the next step.

Creating required resources

The next step is to create the App Service Plan and the Resource Group.

An App Service Plan, in two words, at a very high level is just the container in which your web applications run. It is used to determine the resources available to your application (or applications) and their boundary. Comparing this to an on premises environment, the app service environment is one server or servers on which your application is deployed. The app service plan defines what configuration of hardware your app runs on and how many servers you have. You can deploy multiple web applications inside the same App Service Plan. You do not need a separate web app plan for each web app.

When you create an App Service plan in a certain region (for example, Western Europe), a set of compute resources is created for that plan in that region. Whatever apps you put into this App Service plan run on these compute resources as defined by your App Service plan.

A resource group is a logical container into which Azure resources like web apps, databases, and storage accounts are deployed and managed. The Resource Group can include all the resources for the solution, or only those resources that you want to manage as a group. You decide how to allocate resources to resource groups based on what makes the most sense for your organization. Generally, add resources that share the same lifecycle to the same resource group so you can easily deploy, update, and delete them as a group.

Steps to create App Service Plan with Resource Group simultaneously:

Type App Service Plan in the search bar, select it from the dropdown and click +Create new.

Now you need to configure it. As you don’t have the Resource Group yet, press create new, enter its name, and it will be created with the App Service Plan and placed inside it. Some other tips:
Enter the App Service Plan name that will clearly describe group of resources inside it (for example, name of the project or of the group of projects) select the region (the better option is to choose the closest region to your customers' location because it will affect the latency)
Select operating system to run your apps, in this case Linux was selected
Select the pricing tier. You can choose from dev/test or production plans. But in case you want to configure automatic deployment from GitHub, choose the P1V2 or higher pricing tier. The application plans less than P1V2 have a list of limitations. The next features are not available:

a) Auto-scale feature. Autoscale is a built-in feature of Cloud Services, Mobile Services, Virtual Machine Scale Sets, and Websites that helps applications perform their best when demand changes.
b) Daily backups. Here, probably, nothing to explain.
c) Traffic manager improves performance and availability by routing traffic between multiple instances of your app.
d) Staging slots. Useful for testing and deployments before swapping them into production.

If you want to create a high volume of resources in the future, use tags. Azure tags are name-value pairs that are used to organize resources in Azure Portal. You can apply tags for individual resources or tag the resource group that they are part of.

Press Review and create. It can take up to 10 minutes.

App Service Plan and the Resource Group are ready to place resources inside. That’s why we can proceed to the next steps.

Creating instances for applications

As PaaS was selected, we should search among its services. The appropriate service is App Service. It enables you to build and host web apps, mobile back ends, and RESTful APIs in the programming language of your choice without managing infrastructure. It offers auto-scaling and high availability, supports both Windows and Linux, and enables automated deployments from GitHub, Azure DevOps, or any Git repo.

To create the App Service find this category in the search bar and press +Create.
You will be redirected to this service configuration page. You need to select from the dropdown the subscription on the resource group, where it needs to be located. Put the name of the application (for example my-project-back-end-development).
In the Publish tab select Code option if you are going to just put the code of your project instead of the ready Docker container. It will make Azure create the Docker container for you itself and run your application inside it.
Select Node from the dropdown of runtime stack.
The application region and operating system will be chosen automatically, according to your App Service Plan settings.
Press next and in the Monitoring section I fully recommend you to switch on Application Insights. It allows you to automatically monitor your application. It will automatically detect performance anomalies and includes powerful analytics tools to diagnose issues and to understand how users actually use your app. It's designed to help you continuously improve performance and usability. It works for apps on various platforms, including .NET, Node.js, Java, and Python hosted on-premises, hybrid, or any public cloud. It integrates with your DevOps process and has connection points to various development tools. All other settings, at this moment, let's keep by default. Click create and review, check if all settings are correct, and confirm. It can take up to 5-10 minutes.

The instance for hosting the back-end is ready, and you can reproduce previous steps to create a separate App Service instance for the front-end application. Once it is ready we go to the next part.

Creating instances for database

As mentioned, MongoDB is used in the project. Azure has an opportunity to place MongoDB using the Azure Cosmos DB's API for MongoDB. It has some limitations. With the full list of supported operators and limitations or exceptions you can familiarize by the link; To create DB go to Azure Cosmos DB Resource and click on Create Azure Cosmos DB account. On the next screen select Azure Cosmos DB API for MongoDB.

The instance for hosting the database is ready, and you As on previous steps, select subscription and resource group, enter the database name. Select the location closest to the rest of your resources and customers to decrease the requests time. Select the Mongo version.

Choosing the capacity mode of the instance is an important step to save your money. You need to know the differences between these two options. Provisioned has throughput at a constant rate, regardless of whether operations are being performed on our Cosmos DB account. Serverless Mode — we only use throughput when operations are performed on our Cosmos DB resources. Each of these options has pros and cons and if you want to know more, follow the link. But in two words, serverless — pay only when using, provisioned — reserve capacity for the instance and pay for it all the time. I need to highlight that it is cheaper to use the serverless mode for non-highly loaded projects. Press the create button and wait until it is ready.

Summary

Let’s check what we have at the moment. You can find all the created instances in your resource group on Azure Portal. Each of the application instances are already available with the Microsoft example page by the url, which you can find in the app essentials section. Cosmos db account is ready for moving your database into it.

Now all resources are ready to deploy your applications and database. The next part of the article is going to be about the migration processes and how to connect all instances. Make sure you read both parts of the article and then proceed to work because we will talk about database migration in the next steps.

Code Complete: Is It a Big White Elephant on Your Working Desk?

Mitya Dementiy — Tue, 07 Dec 2021 09:59:18 +0000

When I tweeted about my intention to read Code Complete, I got a couple of warning replies. I found out that the book is "a waste of time", "big white elephant on your desk" and "a dull book to while away the time in one's old age".

But such feedback didn't disturb me because Code Complete is listed in books to read by Hexlet School of Witchcraft and Wizardry where I studied programming.

The book was easy and fun to read. Yes, McConnel's humor is fire. So I laughed many times while reading a serious work about programming. But first thing's first.

The great and powerful Code Complete

The book received Jolt Award in 1993, a prestigious award by Dr. Dobb's Journal for books on software development.

It also received positive reviews from Martin Fowler, Grady Booch, and other famous software engineers and computer science experts.

Hardback version of 900 pages weighs more than 1 kg. Probably, that is what frightens people away from this work. Let's open the book and find out if there is anything to be afraid of?

Russian-language version of Code Copmplete by McConnell

What McConnell wrote in 1993 and reasons to read it in the 21st century

Code Complete is still relevant even though it was first published almost 30 years ago. This book contains very little information that could ever get out of date. McConell discusses fundamental coding principles that are unlikely to change within the next few years.

The code examples are mainly in Visual Basic, Java and C++. But the book's main points make sense regardless of specific languages.

There is a difference between programming into the language and in It.

Programming in your language limits your programming thinking only to the constructs that the language directly supports. If the language tools are primitive, the programmer's thought will also be primitive. Programmers who program "into" a language first decide what thoughts they want to express, and then they determine how to express those thoughts using the tools provided by their specific language. — Steve McConnel.

I had the feeling that there is not that much code in Code Complete.
The author talks about coding in general and occasionally goes to specifics and illustrates his points with examples.

He pays big attention to the process of development — from requirements gathering and choosing the technology to coding, testing, debugging, refactoring, and optimization.

Metaphor is a programmer’s weapon

The book starts with an explanation of why metaphors are important. One of the metaphors offered is building construction.

Metaphors help both developers and any people they work with, such as managers, designers, and clients.

Mistakes that cost the most

They occur on the stage of requirements gathering and designing.

It is easy to explain with building construction metaphors. If you make mistakes when laying the foundation, it will be difficult to fix them when the building is nine floors high.

"Dirty" problems

These are the problems that can be defined only by partially or fully fixing them. I bet every programmer had a situation when a manager asked you to define the time needed for your task. And you explain that it is impossible because you actually have to solve the task first. This is what McConnel calls a dirty problem.

Managing complexity

Managing complexity is the main technical aspect of software development. McConnel quotes Dijkstra, who pointed out that no one's skull is really big enough to contain a modern computer program. This means that software developers shouldn't try to cram whole programs into our skulls at once. We should try to organize our programs in such a way that we can safely focus on one part of it at a time.

Software correctness and software robustness have an inverse relationship

This was a huge insight for me.

For some software robustness is more important than correctness, for some conversely.

For instance, safety-critical software for a spaceship would require a high level of correctness. And we would have to sacrifice correctness in favour of robustness.

Please, leave comments about your interpretation of this point and if you agree with it.

WISCA phenomenon

Why Isn’t Sam Coding Anything? Non-tech people suppose that programmers must be coding nonstop. Clients and bosses get nervous while programmers are planning or working on requirements.

McConnel gives us an easy way to fight WISCA phenomenon. Just keep a straight face and keep the source-code editor window open.

By the way, on the sense of humor. McConnell inserts jokes on nearly every page. They are witty and specific. This is the famous programming humor in its purest form.

What I liked and disliked about the book

I found the book helpful because it’s all about fundamentals. It helps to look for the forests before we look for the trees. And generalize problems and tasks.

The book sure contains plenty of practical and specific examples or recommendations. The parts about variable naming, operators and mastery of programming are pure concentrated usefulness.

I was happy that a big and serious book is written in easy language. Even examples in unfamiliar languages turned out to be comprehendible and useful.

What I didn’t like is how quickly I finished the book. I also didn’t like the chapter on programming tools.The choice of compiler, IDE and other tools depends on which specific technology is used.

To read or not to read?

Well, my conclusion is predictable. The book is worth the money and the time spent. It is more about basics in general rather than a handbook with practical guidelines. Thanks to Code Complete I gained a deeper understanding of coding in general. I started to see things that were out of my attention.

Don’t be put off by the thickness of the book. I guarantee you’ll let out a chuckle many times when reading it.

How to write a super simple reactive state manager

Kapelianovych Yevhen — Wed, 03 Nov 2021 08:24:53 +0000

Every application needs a state management system to have the ability to react to changes in the data. There are lots of state managers for every taste, from easy to understand ones to mind-breaking.

Do you know how they work? What principles stand behind them? I'm sure you are. But these questions I asked myself not a long time ago, and in my opinion, it is still unknown territory for beginners. So, shall we go in?

Behind most managers stands the Observer pattern. It is a powerful pattern. It says that there is a subject - a particular object encloses some data, and there are observers - objects that want to know when that data changes and what value it has now.

How will they know about the change? The subject should tell them that he is changed. For that, every observer should ask the subject to notify it when something happens. It is a subscription.

And when some data changes, the subject notifies all known observers about that. That is a notification.

Pretty simple, yeah?

Practically, there are many implementations for this pattern. We are going to show the simplest one.

Basically, the data of your application aggregates into a restricted scope. In JavaScript, we can use an object for that purpose. Each key represents a separated independent chunk of the data.

const state = {
    key1: "some useful data",
    key2: "other useful data",
    // and so on
}

We can freely read and change these chunks as we want. But the problem is that we cannot predict when the change happens and what piece is changed with what value. Simply put, the object isn't reactive. Fortunately, JavaScript has a feature that helps us track any action that is made with any object. Its name is Proxy.

Proxy is a wrapper around the object which can intercept and redefine fundamental operations for that object (MDN resource).

By default, Proxy passes through all operations to the target object. To intercept them, you need to define traps. A trap is a function whose responsibility is to redefine some operation.

All operations and their trap names you can find [here].(https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Proxy/Proxy#handler_functions)

With this ability, we can write our initial store function. In the end, we should be able to do this:

const appState = store({ data: 'value' });

// Subscribe to the data changes.
appState.on('data', (newValue) => {
    // do something with a newValue
});

// Somewhere in the code
appState.data = 'updated value'; // observer is invoked

As I said earlier, the subject (our object with some data) should notify observers (some entities) when its data was changed. That can be made only when the subject knows what entities want to receive notifications. That means that the subject should have a list of observers inside.

const store = (target) => {
    const observers = [];

    return new Proxy(target, {});
}

And now, we should define a trap for assigning a new value to the target object. That behaviour defines a set interceptor.

const store = (target) => {
    const observers = [];

    return new Proxy(target, {
        set: (target, property, value) => {
            target[property] = value;
            observers
                .filter(({ key }) => key === property)
                .forEach(({ observer }) => observer(value));
            return true;
        },
    });
}

After updating the value, the subject notifies all observers that were added to the list of observers. Great! We've created a notification behaviour. But how does the subject add an observer to the subscription list?

The answer is that the subject should expose a way to trigger this subscription. With Proxy in mind, we can define a virtual method that will accomplish that process. How can we do that?

Virtual method is a method that doesn't exist in the target object, but Proxy emulates it by creating it outside of the target object.

As we know, a method is a property which value is a function. That tells us that we should define a get interceptor and provide a handler for an absent property. At the same time, we shouldn't block access to the target's properties.

const store = (target) => {
    const observers = [];

    return new Proxy(target, {
        get: (target, property) => 
            property === 'subscribe'
                ? (key, observer) => {
                      const index = observers.push({ key, observer });
                      return () => (observers[index] = undefined);
                  }
                : target[property],
        set: (target, property, value) => {
            target[property] = value;
            observers
                .filter(({ key }) => key === property)
                .forEach(({ observer }) => observer(value));
            return true;
        },
    });
}

You may notice that the execution of the subscribe function returns another function. Yes, indeed. Observers should be able to stop listening to changes when they want to. That's why subscribe returns a function that will delete the listener.

And that's it! We may want to make deleting a property reactive. As we did earlier, a delete interceptor is for that.

const store = (target) => {
    const observers = [];

    return new Proxy(target, {
        get: (target, property) => 
            property === 'subscribe'
                ? (key, observer) => {
                      const index = observers.push({ key, observer });
                      return () => (observers[index] = undefined);
                  }
                : target[property],
        set: (target, property, value) => {
            target[property] = value;
            observers
                .filter(({ key }) => key === property)
                .forEach(({ observer }) => observer(value));
            return true;
        },
        deleteProperty: (target, property) => {
            delete target[property];
            observers
                .filter(({ key }) => key === property)
                .forEach(({ observer }) => observer(undefined));
            return true;
        },
    });
}

And now our store function is complete. There are a lot of places for improvements and enhancements. And it is up to you! 🤗

Also, you can see a slightly better implementation in our @halo/store package. A code from these examples lives in the store.js file. But there is one more entity that is worth explaining. That's why we plan to write the next article precisely about it where we are going to explain the purpose of the package and in what situations you may need it. Hold tight and cheer up!

Supported by Halo Lab design-driven development agency

Kickin - bootstrap your Eleventy project

Kapelianovych Yevhen — Mon, 18 Oct 2021 16:55:52 +0000

Make your web pages fast on all devices - Developers all over the world.

We always chase productivity on our pages. And Web development community has got many instruments that help achieve this goal. Eleventy is one of them. It is a great tool that offers an easy way of combining templates and data into fully operative static pages that you can host on a server.

Unfortunately, it can bundle only templates and doesn't care about assets on the site, such as images, styles and scripts etc. So we decided to create a bundler based on Eleventy CLI 🤖
It isn't separate, but we use Eleventy's events for integration.

The primary tool is Kickin CLI. It quickly bootstraps a template folder with Eleventy and necessary plugins for comfortable development. Some of them were built by ourselves:

eleventy-plugin-compress
eleventy-plugin-pwa-icons
eleventy-plugin-scripts
eleventy-plugin-styles
eleventy-plugin-workbox
eleventy-shortcode-image

We don't recommend installing CLI globally. Instead, use a npx for project initialization.

The minimal required version of NodeJS is 12.17.0. The reason is that Kickin CLI consists of ES modules.

It exposes command that is named after itself - kickin. And its use is pretty simple.

npx kickin name-of-your-project

If you already have such a folder, you can use the . symbol instead of the name.

cd my-project-name
npx kickin .

Wait until the installation of dependencies completes, and that's all 🤗

After

There are four main commands available in the created project:

npm start - starts development server on http://localhost:3000
npm run build - builds project.
npm run serve - runs a local server with secure protocol on https://localhost:3000. It helps to mimic the production environment.
npm run certs - creates certificates for the ability to run site with HTTPS locally.

We use mkcert to generate SSL certificates. Make sure you installed it on your local machine as well.

You can change any part of the generated project as you want - configuration file and all dependencies are open to being changed.

Let's explore a little bit of what and where CLI installs.

Structure

Source code is located in the src folder.
- assets: this folder supposed to be home for static content: images, fonts, video, audio etc.
  - images
  - fonts
  - video
  - audio
- components - folder where Eleventy's components are located (default name is _includes)
- layouts - folder for layouts.
- pages - place where future site's pages live.
- scripts - folder for scripts that will be used in HTML.
- styles - folder for styles.
For bundled code, the build folder is preserved.

Inner structure of defined directories is up to you 🙃

Let's dig into plugins a little bit. There are 6 of them.

eleventy-plugin-compress

This plugin is used to compress HTML, CSS and JS using brotli(default), gzip or deflate algorithms. It helps you reduce the size of files that are sent over the internet.

Why brotli?

eleventy-plugin-pwa-icons

If you want to create PWA, you will need icons for your application for different platforms. This package uses pwa-assets-generator to create icons from an image. It will automatically insert links to icons into the HTML of every page and in the manifest.json file along with generated icons.
All you need is an image as a template for future icons. By default, it is a favicon.png under the src directory.

eleventy-plugin-scripts

Starter supports modern JavaScript syntax (except for Eleventy's templates) and TypeScript.

All scripts files should be in the scripts/ directory, but the inner structure does not matter.

To include scripts into an HTML page, provide scripts property in page template's front matter zone (can be either type of string or array of strings). The path to the script is relative to src/scripts directory, so you do not need to include this prefix in the URI.

// For JavaScript templates
module.exports.data = {
  // This will be resolved to 'src/scripts/main.js'
  scripts: 'main.js', // or 'main.ts' -> 'src/scripts/main.ts'
};

eleventy-plugin-styles

The configuration of this one is very similar to eleventy-plugin-scripts.

In Kickin Sass language is supported.

Stylesheets should be in styles/ directory, but the inner structure does not matter. They will be compiled, minified and cleaned from unused selectors automatically.

To include style to page in the template, provide styles property in front matter zone. It can be string or array of strings. The path to the stylesheet is relative to src/styles directory, so you do not need to include this prefix in the URI.

// For JavaScript templates
module.exports.data = {
  // Bundler of this starter is smart enough to recognize type of style
  // and its location (file will be resolved to `src/styles/home.scss`)
  styles: 'home.scss',
};

If you want to put your CSS file in the inner folder, say directory/common.scss , don't forget to mention it in your config:

// For JavaScript templates
module.exports.data = {
  styles: 'directory/common.scss', // -> src/styles/directory/common.scss
};

It is the most potent package of all! Whew... Amazing!

eleventy-plugin-workbox

Adds the Service worker that will automatically cache all assets of your site on a client.

Registration of service worker script will be automatically injected into each HTML page.

By default, the worker will cache static resources (images, fonts, audio files etc.) and try to use them from the cache at first. Dynamic resources that can be changed often (HTML, JS, CSS) are cached also, but on every request, it will try to fetch it on the server at first.

You can read more about strategies here.

eleventy-shortcode-image

It handles images of your page. It can properly rebase, minify and generate optimal types of images that all browsers support. It uses @11ty/eleventy-img under the hood but also can minify SVGs (uses SVGO). Compression is disabled in development mode to reduce build time. You want a quick response on your changes, aren't you? 😊 So, we do too.

For this purpose, the CLI includes an image shortcode. Use it to incorporate images to HTML (either raster or vector(SVG) ones).

Info about shortcodes and how to use them is here.

Signature of shortcode:

interface ImageAttributes {
  /** Defines alternative text for image. */
  alt?: string;
  /** Defines title for image. */
  title?: string;
  /** Class name(s) for `class` attribute. */
  classes?: string | ReadonlyArray<string>;
}

async function image(
  src: string,
  attributes?: ImageAttributes
): Promise<string>;

The first parameter is mandatory and contains an image path relative to the src/assets/images directory.

// For JavaScript templates
module.exports.render = async function () {
  return /* html */ `<main>${await this.image('logo.png')}</main>`;
};

For example, logo.png image will be resolved to src/assets/images/logo.png, then webp and avif images will be generated and placed along with the source image in the build directory. In HTML, you will receive efficient images with the correct paths 😊!

The same rules apply if you define a path to the image from CSS (for example, in the URL function). But if you specify URL as absolute (absolute public URL), it will be returned as-is without any changes.

This is done intentionally in case if you want to copy assets to the output directory.

All these packages are attached to transform flow of Eleventy.

It looks just like above. Every plugin transforms the HTML in its way and could produce new assets. Assets flow separated from Eleventy, but they are synchronised.

Configuration

The default configuration of Eleventy is enough for most use cases, but sometimes there you need to change some behaviour.

.eleventy.js is a starting point for Eleventy.

If you want to change some behaviour or add new feature to site, place it in .eleventy/ directory. It has following structure:

filters - used for holding custom filters.
plugins - contains custom plugins for Eleventy.
shortcodes - contains shortcodes
transforms - holds HTML transformer functions that changes generated HMTL by Eleventy.

Feel free to try it and give us feedback. We fell in love with these tools, and we hope you will too 🖤

Useful links:

This is a repository where a code of all plugins live.

Have fun! ✌️

Supported by Halo Lab design-driven development agency

Implementing the translate function with TypeScript

Kapelianovych Yevhen — Sun, 03 Oct 2021 15:09:31 +0000

TypeScript 4.1 has brought a nice feature called Template literal types. Many authors have written about it, so I won't stop here and explain how it works. Instead, I would like to show you an example of how we can use it. Let's not waste our time!

Every i18n module has a translate function (also can be named as short t). Assume that we have a function with such a signature:

declare function translate(path: string, variables?: Record<string, string>): string;

As the first parameter, we can pass a dot delimited keys of a translation object. It can be a JSON object or JavaScript object, whatever. The main thing is that TypeScript can't check path correctness and possible variable existence in the translated sentence with such signature. Besides, you must be careful with what you are writing. Thanks to the new feature we can fix it.

At first, we need to have a type that describes our translation object. It is up to you because no one knows how big and diversified it can be. Suppose we have it already. And our translation function should know it. Otherwise, TypeScript can't make any prediction on unknown object shapes.

declare function translate<T>(path: string, variables?: Record<string, string>): string;

But it isn't helpful yet. Let's move on. Our goal is to let TypeScript check if keys of the path parameter exist in the translation object and infer the following keys based on typed information.

We can accomplish it by creating all possible ways to structure a path to all values in the object. It may be more clear if I show it with an example.

interface T {
  a: {
     b: string;
     c: string;
    };
  d: string;
}

We have an interface T. For an object with such shape, we can build three possible path parameters:

a.b
a.c
d

And when you write a, TypeScript will add a hint that the following path part can be either .b or .c. Do you get the idea?

But how we can get all possible key combinations? We can recursively iterate over an object type and, for all top keys, get an array of all keys of inner objects that we can reach. It looks like the following:

type PathKeys<T> = T extends string ? [] : {
 [K in keyof T]: [K, ...PathKeys<T[K]>]
}[keyof T];

Stop a little bit and try to understand the type. It is a little bit tricky but works as expected. Can you see why?

Ok, we have tuples of all possible key combinations. But it isn't what we want. So we should join them to have a union type of all combinations. For that, we should learn a new type that will concatenate the values of each tuple. Let's call it Join 😎

type Join<T extends string[], Delimiter extends string> =
 T extends [] ? never :
 T extends [infer F] ? F :
 T extends [infer F, ...infer Other] ?
 F extends string ?
 `${F}${Delimiter}${Join<Extract<Other, string[]>, Delimiter>}` : never : string;

It is a bit trickier than the previous type, and we finally start using the new TypeScript feature. It accepts an array of strings (keys) and delimiter value that is the dot in our case. Then recursively iterates over the array(tuple). Unfortunately, TypeScript can't understand that the Other type is another tuple with all values except the F and infers it as unknown[], but we can fix it by providing a hack with an Extract type that narrow Other to a tuple of strings.

And that's it! Now we should make path parameter as union type of all keys combinations.

interface T {}

declare function translate<P extends Join<PathKeys<T>, '.'>>(paths: P, variables?: Record<string, string>): string;

Yay!

But one thing left. Our text can have variables, and we can provide values for them. Are we able to let TypeScript check for variable presence in translation sentences and check for name correctness? Yes, we are.

To do that, we should analyze a string and check whether it has a particular template for a variable. Suppose it is {variableName}. As the previous types, the new type will also be recursive.

type SearchForVariable<A extends string> = A extends `${infer A}{${infer B}}${infer C}`
 ? SearchForVariable<A> | B | SearchForVariable<C>
 : never;

We split a string into three parts: before a template, template itself and after the template. Returned union type will contain all variable names that exist in the string value or nothing.

But how we actually can get the correct value for checking? For that, we should match the initial translation object and path to the string value. Do you know what it means? Yes, a new type! Let's dig into it.

type Variables<T extends string | object, Path extends string, Delimiter extends string> =
 Path extends `${infer A}${Delimiter}${infer O}`
 ? A extends keyof T
 ? Variables<Extract<T[A], string | object>, O, Delimiter>
 : never
 : Path extends `${infer A}`
 ? A extends keyof T
 ? SearchForVariable<Extract<T[A], string>>
 : never
 : never;

It is the trickiest type of this tutorial. At first, we should get rid of the delimiter. Then every part of the inferred values is recursively checked. If at least one part contains a variable template, then it is returned. If there are two variables or more, then the union type of their names is the result. So, the final type of translate function looks like this:

interface T {}

declare function translate<P extends Join<PathKeys<T>, '.'>>(paths: P, variables?: Record<Variables<T, P, '.'>, string>): string;

We have built a complete translate function that TypeScript can check for correctness. You must have a concrete translation type object with values as literal string types for it to work. Otherwise, TypeScript won't make any checks 🤷‍♂️.

As a bonus, you can allow spaces between {} and variable names. But in that case, you should trim extra whitespaces from the name. A new type called Trim can help you with that.

type Trim<A extends string> = A extends ` ${infer B}` ? Trim<B> : A extends `${infer C} ` ? Trim<C> : A;

Also, you should correct the SearchForVariable type to perform the trim operation over the variable's name.

type SearchForVariable<A extends string> = A extends `${infer A}{${infer B}}${infer C}`
 ? SearchForVariable<A> | Trim<B> | SearchForVariable<C>
 : never;

A small update. And now we are finally done. The final implementation you can see here.

Thank you for reading. TypeScript is a great language and gets more power with each release. I hope you had fun and learned something new. May the wisdom be with you.

Supported by Halo Lab design-driven development agency