DEV Community: Yues IT

[YuIT Docs📜]⚛️FrontEnd System Design

Yues IT — Sun, 16 Nov 2025 16:55:56 +0000

All source files and library 👉in this repo

Let's see some approaches for authorization of an app via FrontEnd System Design using the following technologies:

Keycloak is Open Source Identity and Access Management. Add authentication to applications and secure services with minimum effort. No need to deal with storing users or authenticating users.
Gogatekeeper is a proxy which integrates with Keycloak IDP Provider, it supports both access tokens in a browser cookie or bearer tokens.
Keycloak JS is secure web applications. The adapter uses OpenID Connect protocol under the covers. You can take a look at the Securing applications and services with OpenID Connect guide for the more generic information about OpenID Connect endpoints and capabilities. The specific guide for this client

Let's drawing those approaches via Component design

1️⃣ Component diagram for authorization via Gogatekeeper + Keycloak

2️⃣ Component diagram for authorization via Keycloak.js

[YuIT Docs📜]📐System Design

Yues IT — Sat, 12 Jul 2025 13:12:02 +0000

All source files and library 👉in this repo

What is System Design?

System design is the process of developing and planning the architecture, components, modules, and interfaces for creating complex software, hardware, or information systems.

There is the good tutorial about System Design in here

In this article I want to show how to apply SD in practice -> step by step.

Let's try to create a system design for the Dating Service (analog of Tinder, Bumble, Badoo, etc.)

1️⃣ Requirements Gathering Process.
In this step, let's consider Functional vs. Non Functional Requirements

Functional requirements:

the user can register in the application

the user can create/edit a personal profile

users can receive notifications when receiving dating offers or when two users are automatically matched

the user can accept/reject dating offers

the user can view a list of other users according to their preferences and their location

the user can upload photos/small videos

administrators should have their own functionality for moderation

Non-functional requirements:

System responsiveness (so that the user can quickly view the list of potential candidates for dating)

High availability (as the load increases, the application must scale and be available, otherwise users will go to another similar service)

Consistency (when accepting/rejecting offers between users)

Provide monitoring with alerts for the system

2️⃣ Capacity Estimation

Metrics for Capacity Estimation

100M - Monthly Active Users (MAU)

10M - Daily Active Users (DAU)

about 10M - registered users (REG_USERS)

about 3MB(megabyte) - user data (USER_DATA): messages, photos/small videos, profile data, etc.

let's assume that the ratio of writing to reading data is 1 to 10 (RATIO_WRITE_READ)

on average one user performs about 50 operations (OPERS)

86400 - number of seconds in a day (NUM_SEC_DAY)

365 - number of days in the year (DAYS_OF_YAER)

Estimated load indicators
Requests Per Second (RPS) or Queries Per Second (QPS):

RPS_READ = OPERS*REG_USERS/NUM_SEC_DAY = 50*10M/86400 = 5800 - RPS for reading

RPS_WRITE = RPS_READ/RATIO_WRITE_READ = 5800/10 = 580 - RPS for writing data

Network Bandwidth (NB)

NB = USER_DATA*RPS_WRITE = 3MB*580 ~ 2000 MB/sec (2GB/sec)

NB_PER_YEAR = NB*NUM_SEC_DAY*DAYS_OF_YAER = 2GB/sec * 86400sec * 365days ~ 65PB + 20% (with Overhead) ~ 80PB(petabyte)

Storage load (SL)

SL = DAU*USER_DATA = 10M*3MB = 30TB(terabyte) ~ 30TB + 20% (with Overhead) ~ 40TB

Calculation of cloud infrastructure capacity

let's assume that one instance can handle with about 500 RPS: NUMBER_INST = RPS_READ(Peak RPS)/500 = 5800/500 + 20%(with Overhead) ~ 15 instances;

Instances for Network Bandwidth: Convert to Gbps (125 MB/sec = 1 Gbps) NB = 2000/125 ~ 16 Gbps; if each instance can handle 1 Gbps, then NUMBER_INST_NB = 16/1 ~ 20 instances (with Overhead);

Storage, if each storage instance can handle 5TB of data, then NUMBER_INST_STOR = SL/5 = 40/5 = 8 instances. If you implement 2x replication for fault tolerance, you need twice the storage: 8*2 = 16 ~ 20 instances (with Overhead);

How much does it cost?💵

Network Bandwidth (~0.1$ for 1GB): 80PB = 80,000,000GB*0.1$ = 8M$

SSD storage (~300$ for 1TB): 40TB * 300$ = 12,000$

3️⃣ High-Level Design

By high-level design we will mean a design that minimizes the basic scenarios of interaction with the system, but does not reflect all the complexity of interaction between the large number of real components and does not take into account the load, under which the system will be in a productive environment.

In this step, let's draw the scheme of components for our system using Excalidraw.

To utilize the following components of SD

From this library

Through those components, let's build the next scheme.

4️⃣ Component design
Previously, we looked at the high-level components that most systems will consist of.
At the same time, we often hid all the logic inside the application behind one logical "square". Let's try to delve into the details of what can actually happen under the hood.
And we can separate our scheme into two parts:

The first part is intended for users
The second part is intended for app administrators

5️⃣ Scalability and reliability

6️⃣ Finally, let's consider Monitoring and Security
Detail you can see in this

[YuIT Docs📜] Create a new SSH key🔑

Yues IT — Thu, 08 May 2025 15:49:20 +0000

Create a new SSH connection between your local machine and GitHub

1️⃣ Open the Terminal of VScode and generate a New SSH Key

ssh-keygen -t ed25519 -C "my-key"
- Enter file in which to save the key (C:\Users\someuser\.ssh\id_ed25519): 
- C:\Users\someuser\.ssh\id_ed25519 already exists.
- Overwrite (y/n)? y
- Enter passphrase for "C:\Users\someuser\.ssh\id_ed25519" (empty for no passphrase):
- Enter same passphrase again:
- Your identification has been saved in C:\Users\someuser\.ssh\id_ed25519

2️⃣ Go to our directory with private and public keys:

cd ~/.ssh

3️⃣ Copy the SSH Public Key

For windows

cat id_ed25519.pub | clip

For macos

cat id_ed25519.pub | pbcopy

4️⃣ Add the SSH Key to Your GitHub Account

Log in to GitHub
Go to Settings > SSH and GPG keys
Click New SSH key
Paste your key into the "Key" field
Add a descriptive title
Click Add SSH key

Or add the SSH Key to Your GitLab Account

Sign in to GitLab
On the left sidebar, select your avatar
Select Edit profile
On the left sidebar, select SSH Keys
Select Add new key
Select Key, and you should see the 1Password helper appear
Select the 1Password icon and unlock 1Password.
You can then select Create SSH Key or select an existing SSH key to fill in the public key

5️⃣ Test the SSH Connection

ssh -T git@github.com

ssh -T git@gitlab.com

Summary:

- Generate SSH key (ssh-keygen)
- Add key to SSH agent (ssh-add)
- Copy public key (cat ~/.ssh/id_ed25519.pub) and add to GitHub or Gitlab
- Test connection (ssh -T git@github.com or ssh -T git@gitlab.com) 
- Use SSH URL for repositories

Guide to add a file, commit your changes, and prepare for pushing to GitHub:

Step 1. Clone your Repositories

cd C:/MyPath/GitHub/
git clone git@github.com:username/repository.git

Step 2. Add your new or modified files

To stage a specific file:

git add filename

To stage all changed files:

git add .

Step 3. Commit your changes with a descriptive message:

git commit -m "Your commit message"

Step 4. Push your changes to GitHub

git push origin main

ssh

Yues IT — Thu, 08 May 2025 14:56:59 +0000

[YuIT Docs📜] GitLab🦊 Runner

Yues IT — Mon, 10 Mar 2025 20:00:42 +0000

Hello!👋 In this article, we can try to register a new gitlab runner in a docker container📦 via:

Your project on gitlab.com
Docker desktop

Before your getting started, you can read the official documentation and don't read this article 😉

So, if you're still here, let's do it together!💪

1️⃣ In you local or virtual machine, you should do those steps:

Pull the GitLab Runner Docker Image: Open your terminal and run the following command to pull the GitLab Runner Docker image: docker pull gitlab/gitlab-runner:latest
Create new volume, it's a good practice to use a Docker volume to persist the config.toml file and ensure it remains available between container restarts. You can create a volume using: docker volume create gitlab-runner-config, where:
- Names the volume: gitlab-runner-config
Run the GitLab Runner Container:
Next, run the GitLab Runner container. You can use the following command to start it in detached mode: docker run -d --name yuit-gitlab-runner --restart always -v gitlab-runner-config:/etc/gitlab-runner -v /var/run/docker.sock:/var/run/docker.sock gitlab/gitlab-runner:latest
This command does the following:
- Runs the container in detached mode: -d.
- Names the container yuit-gitlab-runner.
- Restarts the runner automatically if it stops: --restart always.
- Maps the configuration directory via: -v gitlab-runner-config:/etc/gitlab-runner: This command mounts a Docker volume named gitlab-runner-config into the filesystem of the container at the specified path (/etc/gitlab-runner).
- Add docker socket: -v /var/run/docker.sock:/var/run/docker.sock: This command mounts the host's Docker socket file (/var/run/docker.sock) at the same location inside the container.

Explanation of Docker Socket: By mounting the Docker socket, the GitLab Runner can use Docker to run builds as containers. This is necessary for scenarios where your CI/CD pipelines leverage Docker as an executor for running jobs, allowing the GitLab Runner to create and manage Docker containers during the job execution.

2️⃣ Next step is Register runner in gitlab

You have to go to your project in gitlab.com and do those steps:

Select Settings ⟶ CI/CD ⟶ Runners

Select New instance runner.
In the Tags section, in the Tags field, enter the job tags to specify jobs the runner can run, for example, my tag name's is : yuit-runner-tag. If there are no job tags for this runner, select Run untagged jobs

Runner description: Runner of docker container
Select the operating system where GitLab Runner is installed: linux

3️⃣ Move on and you should register runner in local docker container

On your local machine to open terminal and run following command:
docker exec -it yuit-gitlab-runner gitlab-runner register.

In terminal, you need to enter:

the GitLab instance URL: https://gitlab.com
the registration token: ****
the name for the runner: yuit-runner
the executor: docker
the default Docker image: node:20.16-alpine

4️⃣ Go to gitlab.com

In there we can see that our runner is registered
After clicking View runners, we can see that our runner is available

[YuIT Docs📜] 🐳Docker compose for a Full Stack App

Yues IT — Sun, 19 Jan 2025 16:13:12 +0000

Introduction to Docker Compose

🐳Docker Compose is a powerful tool for defining and running multi-container Docker applications.
📑In this article, we will delve into the world of Docker Compose using a real-world example of a web application that consists of a PostgreSQL database, a backend service, and a frontend service.
👉In the previous article I have seen how use only docker for this

What is Docker Compose?

You just can read the official documentation😁
Or read a short description: Docker Compose is a command-line tool that allows you to define and run multi-container Docker applications. It uses a YAML file, known as the docker-compose.yml file, to define the services that make up your application. Docker Compose handles the creation and management of the containers, networks, and volumes required by your application.

Our example application consists of three services:

Database - PostgreSQL: This service is responsible for storing data for our application.
Backend - Nest.js: This service handles the business logic of our application and interacts with the database.
Frontend - Next.js: This service provides a user interface for our application and interacts with the backend service.

Docker Compose Configuration

❗Before we start, we need:

1️⃣ To clone examples repositories for:

BE: git clone https://github.com/YuesIt17/yuit-docs-nestjs-docker.git
FE: git clone https://github.com/YuesIt17/yuit-docs-nextjs-docker.git
Docker Compose: git clone https://github.com/YuesIt17/yuit-docs-docker-config.git

2️⃣ To start the docker db container and create docker images for FE and BE. I describe this in my README:

3️⃣ To run the containers via: docker-compose up -d. If you wanna stop and remove the containers via:docker-compose down

4️⃣ You can then access the application by visiting http://localhost:3000 in your web browser

👀Here is the docker-compose.yml file that defines our example application:

version: "3.8"

services:
  db:
    image: postgres
    container_name: yuit-chart-db-host
    environment:
      POSTGRES_DB: yuit-chart-db
      POSTGRES_USER: admin
      POSTGRES_PASSWORD: adminyuitpassword
    networks:
      - yuit-net
    ports:
      - "5432:5432"
    restart: unless-stopped

  backend:
    image: yuit-chart-backend-image:latest
    container_name: yuit-chart-backend
    networks:
      - yuit-net
    ports:
      - "127.0.0.1:8080:8080"
    depends_on:
      - db
    restart: unless-stopped

  frontend:
    image: yuit-chart-frontend-image:latest
    container_name: yuit-chart-frontend
    networks:
      - yuit-net
    ports:
      - "127.0.0.1:3000:8080"
    restart: unless-stopped

networks:
  yuit-net:
    driver: bridge

💪Let's break down this configuration:

version directive specifies the version of the Docker Compose format.
services directive defines the services that make up our application.
db service uses the official PostgreSQL image and sets environment variables for the database name, user, and password.
backend service uses a custom image (yuit-chart-backend-image:latest) and depends on the db service.
frontend service uses a custom image (yuit-chart-frontend-image:latest) and does not depend on any other services.
networks directive defines a network named yuit-net with a bridge driver.

Key Concepts

Here are some key concepts to understand when working with Docker Compose:

Services: A service is a container that runs a specific image. In our example, we have three services: db, backend, and frontend.
Images: An image is a template for creating containers. In our example, we use the official PostgreSQL image and two custom images.
Networks: A network is a way for services to communicate with each other. In our example, we define a network named yuit-net with a bridge driver.
Environment Variables: Environment variables are used to configure services. In our example, we set environment variables for the database name, user, and password.
Depends On: The depends_on directive specifies that a service depends on another service. In our example, the backend service depends on the db service.

Conclusion

In this article, we introduced Docker Compose. We defined a docker-compose.yml file that specified three services: a PostgreSQL database, a backend service, and a frontend service. We then ran the application using the docker-compose up command. Docker Compose provides a powerful way to define and run multi-container Docker applications, making it an essential tool for developers and DevOps teams.

Thanks for reading and feedback!
See you soon!🤝

[YuIT Docs📜] Docker container for a Full Stack App📦

Yues IT — Wed, 01 Jan 2025 12:36:48 +0000

A Docker container📦 image is a lightweight, standalone, executable package of software that includes everything needed to run an application: code, runtime, system tools, system libraries and settings.

📃In this article we will figure it out, how to deploy your own app using 🐳Docker and 🌐Nginx, for our app we will use the following technologies:

⚛️ For frontend (my repository): Next.js is a popular open-source React framework that enables developers to build server-rendered and static web applications with ease. It offers features like automatic code splitting, server-side rendering (SSR), static site generation (SSG), and API routes, which enhance performance and SEO. Next.js simplifies the development process by providing a file-based routing system and built-in support for CSS and TypeScript, making it a powerful choice for creating modern web applications.
🔚 For backend (my repository): Nest.js is a progressive Node.js framework for building efficient, scalable server-side applications. It leverages TypeScript and incorporates modern design patterns, such as modular architecture and dependency injection, to promote maintainability and testability. With built-in support for GraphQL, WebSockets, and microservices, Nest.js is ideal for developing robust APIs and enterprise-level applications. Its extensible architecture allows integration with various libraries and tools, making it a versatile choice for backend development.
🗄️ For database: PostgreSQL is a powerful, open-source relational database management system (RDBMS) known for its robustness, extensibility, and compliance with SQL standards. It supports advanced data types, complex queries, and transactions, making it suitable for a wide range of applications. With features like ACID compliance, full-text search, and support for JSON and GIS data, PostgreSQL is favored for its performance, reliability, and ability to handle large volumes of data. Its active community and rich ecosystem of extensions further enhance its capabilities, making it a popular choice for developers and organizations.

Sometimes developers confuse Nest and Next😀
Ok, let's move on and do it!💪

📑Create docker network

The fist of all to create a common network for feauture containers
First of all, it is necessary to create and check a common network for future docker containers via command:

docker network create -d bridge yuit-net, where are the following parameters:

yuit-net - the name of our network

bridge - the default network driver

-d or --driver - this flag allows you to specify which network driver docker should use to create the network

docker network ls - for check network

And in our terminal we can see all networks (including ours)

PS ...\yuit-docs-nestjs-docker> docker network create -d bridge yuit-net
6019f719ee8bcd39cf0406a795f5a61a531756ce29c9a53974868d7797ed19e7
PS ...\yuit-docs-nestjs-docker> docker network ls
NETWORK ID     NAME       DRIVER    SCOPE
33c0431165ec   bridge     bridge    local
295269f446db   host       host      local
801fdcaabbb4   none       null      local
6019f719ee8b  yuit-net   bridge    local

📑Create docker container for database

Next step we need to create container for our database PostgreSQL via:

docker run --network=yuit-net --name yuit-chart-db-host --rm -d -p 5432:5432 -i -t -e POSTGRES_DB=yuit-chart-db -e POSTGRES_USER=admin -e POSTGRES_PASSWORD=adminyuitpassword postgres, where are the following parameters:

--network=yuit-net - it's our docker network

--name yuit-chart-db-host - it's a custom name to the container for data source option in our backend project (we will consider this below)

--rm - this option tells docker to automatically remove the container when it stops

-d - this flag runs the container in detached mode

-p 5432:5432 - this option maps port 5432(first) of the host machine to port 5432(second) of the container

-i - this flag stands for "interactive" mode. It keeps the standard input (stdin) open even if not attached, allowing you to interact with the container if needed

-t - this flag allocates a pseudo-TTY (terminal) for the container, which is useful for running interactive applications. It is often used in conjunction with -i.

-e POSTGRES_DB=yuit-chart-db - this option sets an environment variable inside the container. Here, it creates a PostgreSQL database named yuit-chart-db

-e POSTGRES_USER=admin - this environment variable sets the username for the PostgreSQL database

-e POSTGRES_PASSWORD=adminyuitpassword - this environment variable sets the password for the PostgreSQL user specified in the previous parameter

postgres - it's the official PostgreSQL image from Docker Hub

And via command docker ps -a to check the DB container

PS ...\yuit-docs-nestjs-docker> docker run --network=yuit-net --name yuit-chart-db-host --rm -d -p 5432:5432 -i -t -e POSTGRES_DB=yuit-chart-db -e POSTGRES_USER=admin -e POSTGRES_PASSWORD=adminyuitpassword postgres
PS ...\yuit-docs-yues-nestjs-docker>  docker ps -a
CONTAINER ID   IMAGE      COMMAND                  CREATED         STATUS         PORTS                    NAMES
2ef9f2d7daa6   postgres   "docker-entrypoint.s…"   5 seconds ago   Up 5 seconds   0.0.0.0:5432->5432/tcp   yuit-chart-db-host

❔ We may also connect to our DB, for example, via Database Tool: DBeaver

And we see our DB with the name yuit-chart-db

📑Create docker image and container for backend

After we have created our DB, we can create a docker image of backend.
For that:

1️⃣We need to create our Dockerfile, it's needed for describe our image, for example

 # build app
 FROM node:20.16-alpine AS build
 WORKDIR /app
 COPY . /app

 ENV DATABASE_HOST=yuit-chart-db-host

 RUN yarn install
 RUN echo "Build is ok! Go to next layer!"
 RUN yarn build

 EXPOSE 8080

 # start migration and api
 CMD npm run migration:prod && npm run start:prod

2️⃣After that we can build image of backend, via command:

docker build . -t yuit-chart-backend-image -f infra/docker/Dockerfile, where are the following parameters:

the dot (.) - indicates that the current directory should be used as the context for the build

-t yuit-chart-backend-image - it's name of your image

-f infra/docker/Dockerfile - it's directory of our Dockerhfile

3️⃣The next step is run container:

docker run --network=yuit-net --name yuit-chart-backend -d -p 127.0.0.1:8080:8080/tcp yuit-chart-backend-image, where are the following parameters:

--network=yuit-net - it's our docker network

--name yuit-chart-backend - this flag assigns a name to the container, i.e. yuit-chart-backend

-d - this stands for detached mode

-p 127.0.0.1:8080:8080/tcp - this option is used to publish a container's port to the host, where:

8080:8080 indicates that port 8080 on the host will be mapped to port 8080 on the container.

/tcp specifies the protocol to use (in this case, TCP). This part is optional, as TCP is the default protocol

yuit-chart-backend-image - it's the name of the Docker image

❔ We can check our contaner via command docker ps -a or to see containers in the docker desktop

We may also select our data from the DB, because we have run migrations in the Dockerfile. To do this, just run the select command in the DBeaver:

SELECT id, data
FROM public.chart;

SELECT *
FROM public.__migrations;

4️⃣After running the backend container, we can test the api through url: http://localhost:8080/data_chart

📑 Create docker image and container for frontend

Finally, we need to create an image and run a container for the frontend.

1️⃣ Let's add settings for the web server - nginx, for example:

Add these settings to the file conf.d

http {
  server {
    listen  8080;
    server_name localhost;

    location / {
        root /usr/share/nginx/html;
        index index.html;
        try_files $uri /index.html;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }
  }
}

events {}

2️⃣We need to create our Dockerfile, for example:

# build app
FROM node:20.16-alpine as build
WORKDIR /app
COPY . /app
RUN yarn
RUN echo "Build is ok! Go to next layer!"
RUN yarn deploy

# build server
FROM nginx:alpine as normal
# copy the build folder from react to the root of nginx
COPY --from=build /app/build /usr/share/nginx/html
COPY infra/docker/conf.d /etc/nginx/nginx.conf
RUN echo "Nginx is ok!"
# expose port 3000 to the outer world
EXPOSE 3000
# start nginx
CMD ["nginx", "-g", "daemon off;"]

3️⃣After that we can build image, via command:

docker build -t yuit-chart-frontend-image --target normal -f infra/docker/Dockerfile ., where are the following parameters:

-t yuit-chart-backend-image - it's image name

-f infra/docker/Dockerfile - it's directory of our Dockerhfile

--target normal - it's needed for build only a part of the Dockerfile, which can be useful for optimizing the build process and reducing image size

the dot (.) - indicates that the current directory should be used as the context for the build

4️⃣The next step is run container:

docker run --network=yuit-net --name yuit-chart-frontend --rm -itd -p 127.0.0.1:3000:8080/tcp yuit-chart-frontend-image, where are the following parameters:

--network=yuit-net - it's our docker network

--name yuit-chart-frontend - this flag assigns a name to the container

--rm - this option tells Docker to automatically remove the container when it exits

-itd - this stands for i - interactive mode, t - allocates a pseudo-TTY and d - detached mode

-p 127.0.0.1:3000:8080/tcp - this option is used to publish a container's port to the host, where:

3000:8080 indicates that port 3000 on the host will be mapped to port 8080 on the container

/tcp specifies the protocol to use (in this case, TCP). This part is optional, as TCP is the default protocol

yuit-chart-frontend-image - it's the name of the Docker image

5️⃣After running the frontend container, we can test the api through http://localhost:3000/

My examples:

Thanks for reading and feedback!🤝

[YuIT Docs📜] Create a new UI platform using an NPM package📦

Yues IT — Wed, 13 Nov 2024 16:35:28 +0000

📃In this article we will figure it out, how to create your own npm package using the following technologies:

🏗️Rollup.js is a JavaScript module bundler that compiles small pieces of code into larger, more complex applications. It is particularly well-suited for bundling ES6 modules and is known for its efficient tree-shaking capabilities, which eliminate unused code from the final bundle, resulting in smaller file sizes.
📦Lerna.js is a popular tool for managing JavaScript projects with multiple packages, often referred to as monorepos. It simplifies the process of versioning, publishing, and managing dependencies across various packages within a single repository.
🐸JFrog Artifactory is a universal artifact repository manager that enables organizations to store, manage, and distribute software packages and artifacts across the entire development lifecycle. It supports a wide range of package formats, including Docker, Maven, npm, PyPI, and more, making it a versatile solution for DevOps and CI/CD pipelines.

JFrog Artifactory Settings

If you don't have artifactory, you can use the free trial or the free open source version. But the free open source is a cut down version that doesn't have any pre-built settings for the npm repository. Let's do it step by step!

Let's create a new virtual npm repository. It's necessary to install our npm package from external access (public repository).
The JFrog virtual npm repository is a consolidated view of multiple npm repositories, allowing users to access and manage npm packages from various sources in a single location. It acts as a proxy for local and remote npm repositories, enabling seamless integration of public and private packages. This setup simplifies dependency management, enhances performance through caching, and provides a unified interface for package retrieval. Additionally, virtual repositories support security and access control, ensuring that only authorized users can access specific packages.
Add a new local repository. It's necessary to publish our npm package to private repository.
The JFrog local npm repository is a dedicated storage space within JFrog Artifactory for hosting and managing your organization's private npm packages. It allows teams to publish, version, and manage their own npm packages securely and efficiently. By using a local repository, developers can ensure that their packages are readily available, control access permissions, and maintain a consistent versioning system. Local npm repositories also facilitate faster builds and deployments by providing quick access to internal packages, ultimately enhancing the overall development workflow.
Add a new remote repository. It's necessary for to save cache from external registry: https://registry.npmjs.org or https://registry.yarnpkg.com/
The JFrog remote repository is a type of repository in JFrog Artifactory that acts as a proxy for a remote artifact repository, such as npm registry or Docker Hub. It allows users to access and cache artifacts from these external sources without directly interacting with them.
- Caching: Artifacts are cached locally upon first access, improving performance for subsequent requests. Access Control: Provides security and access management for remote artifacts.
- Aggregation: Can aggregate multiple remote repositories into a single endpoint, simplifying dependency management.
- Metadata Management: Supports metadata for better organization and search capabilities.
Let's move on to editing the virtual repository to combine with local and remote ones.

NPM Packages using Rollup.js + Lerna.js

For our NPM packages example we use Rollup.js + Lerna.js (link to my example npm library). As for the usage of these libraries, we can look at the official documentation:

❗But I'd like to show you the details of the settings ⚙.

Rollup settings

After install Rollup.js, i have set up the basic rollup configuration:

import {defineConfig} from 'rollup';
import resolve from '@rollup/plugin-node-resolve';
import commonjs from '@rollup/plugin-commonjs';
import typescript from '@rollup/plugin-typescript';
import {terser} from 'rollup-plugin-terser';
import url from '@rollup/plugin-url';

import * as fs from 'fs';
import path from 'path';

const PACKAGE_NAME = process.cwd();
const packageJson = JSON.parse(fs.readFileSync(path.join(PACKAGE_NAME, 'package.json'), 'utf-8'));

const includePaths = ['**/*.woff', '**/*.woff2', '**/*.svg', '**/*.png'];

export default defineConfig({
  input: 'src/index.ts',
  output: [
    {
      file: packageJson.main,
      format: 'cjs',
      sourcemap: false,
      name: packageJson.name,
    },
    {
      file: packageJson.module,
      format: 'es',
      sourcemap: false,
      name: packageJson.name,
    },
  ],
  plugins: [
    resolve(),
    commonjs(),
    typescript({tsconfig: './tsconfig.json'}),
    terser(),
    url({
      fileName: '[name][extname]',
      include: includePaths,
      limit: 0,
    }),
  ],
  external: [...Object.keys(packageJson.peerDependencies || {})],
});

From this configuration we see the following settings:

packageJson is needed to get settings from the package.json file (for which the build is in progress)
includePaths is needed to support the paths to our images, fonts and ect. We can also save the file name via props fileName, otherwise the file will be an auto-generated name.
external option is used to specify which modules should be treated as external dependencies. ❗When you mark a module as external, Rollup will not include it in the output bundle. Instead, it will assume that the module will be available in the environment where the bundle is executed (e.g., in your project where you will install this npm package). In here we includes peerDependencies from our packages.
Define peerDependencies in package.json, you can specify the libraries that are expected to be installed by the consumer of your package.
For example, we added peer dependencies in the package ui

  "peerDependencies": {
    "react": "^18.x",
    "react-dom": "^18.x"
  }

These peer dependencies ensures the package ui can be installed with the 18.x major version of the package react and react-dom only and these React libraries must be installed from your project (e.g. i installed them in my client project yuit-docs-vite)

⁉In other words when using peerDependencies with the external option:

external ensures when you run the Rollup build command, it will create a bundle that does not include react and react-dom

peerDependencies ensures that these dependencies will be provided by the consumer of your library (i.e. in my case it's yuit-docs-vite) and limits react and react-dom versions to only major version 18.x

Also you can see other official rollup plugins here.

Lerna settings

After init Lerna.js we can see lerna.json:

{
  "$schema": "node_modules/lerna/schemas/lerna-schema.json",
  "version": "independent",
  "packages": ["packages/*"],
  "npmClient": "yarn"
  "command": {
     "publish": {
       "registry": "https://some-art.com/artifactory/api/npm/portfolio-yues-it-npm/"
    }
   }
}

I added npmClientand command for testing lerna publish via yarn, but it didn't work for me and i published via yarn workspaces foreach no-private (we will it below ⬇).
An example of a project structure looks like this (❗Please note that in the settings you need to specify the prefix of the name of our root project):

❗Before to run npx lerna commands you need to set up ssh access to git via terminal (this is necessary for local publishing from a git repository) and test the connection for github: ssh -T git@github.com.

To publish to git, you need to add settings to .yarnrc.yml:

enableStrictSsl: false

httpTimeout: 12000000

nodeLinker: node-modules

npmAlwaysAuth: false

npmPublishRegistry: 'https://some-art.com/artifactory/api/npm/portfolio-yues-it-npm/'

npmRegistryServer: 'https://some-art.com/artifactory/api/npm/portfolio-yues-it-npm/'

npmAuthToken: 'some token from a virtual repository jfrog (Repositories -> Virtual -> Set me up -> Configure -> Generate token)'

yarnPath: .yarn/releases/yarn-berry.cjs

❗Please note:

this url of virtual repository - 'https://some-art.com/artifactory/api/npm/portfolio-yues-it-npm/' we take from artifactory
in addition, if you have a problem with public to JFROG Artifactory (when the yarn uses the Glogal settings), because the yarn doesn't use the local project settings from file .yarnrc.yml -> ❗you can see corepack for isolation yarn configs for different projects.

Once lerna and git are configured we can:

install a npm package for a specific lerna package via: yarn workspace @portfolio-yues-it-npm/utils add date-fns

version in git and publish to artifactory via:

run build via: npx lerna run build

then: yarn install

make commit changes

run versioning via: npx lerna version --no-private (📝--no-private is needed to publish packages other than privates ones)

select the necessary version: Patch / Minor / Major (after that will be push to your git repository)

make publish to artifactory via yarn: yarn workspaces foreach --all --no-private npm publish (⁉when publishing via: npx lerna publish --no-private, sometimes there is a problem after publishing with installing packages, i.e. with slash decoding (instead of '/' to '%2f'), because of which it doesn't find the path to the package and there is the error)

You can see more of my commands in README

How to use npm packages

After setting jfrog and npm project (lerna + rollup) we can install this package.
But before we must:

write virtual url registry of our artifactory in .yarnrc.yml

enableStrictSsl: false

httpTimeout: 600000

nodeLinker: node-modules

npmRegistryServer: 'https://some-art.com/artifactory/api/npm/portfolio-yues-it-npm/'

yarnPath: .yarn/releases/yarn-4.5.1.cjs

than install package via:
- yarn add @portfolio-yues-it-npm/ui
- yarn add @portfolio-yues-it-npm/utils
- yarn add @portfolio-yues-it-npm/icons

❗If you want to test your npm package without publishing, just provide that path (file:../portfolio-yues-it-npm/packages/ui) in dependencies and run yarn install

  "dependencies": {
    "@portfolio-yues-it-npm/ui": "file:../portfolio-yues-it-npm/packages/ui",
    "react": "^18.3.1",
    "react-dom": "^18.3.1"
  },

My examples:

Thanks for reading and feedback!🤝