DEV Community: Duc Tran

Chat UI for Deepseek in your local

Duc Tran — Sat, 01 Feb 2025 08:30:35 +0000

Chat deepseek UI

A simple web-based chat interface powered by Ollama's deep-seek model.
This tool provides an interactive, real-time chatting experience with the
assistant, using Streamlit for the frontend. The assistant processes and streams
responses back to the user, ensuring a seamless and engaging interaction.

Full source code in here

Features

Real-time Chat Interface: Allows for dynamic interaction with the assistant.
Persistent Chat History: Maintains chat history between sessions.
Save & Load Chat History: Automatically saves chat history to a JSON file for persistent conversations.

Setup

Install ollama

Pull deepseek-r1 model

ollama pull deepseek-r1

Clone repo

git clone -b v1 https://github.com/ductnn/chat-deepseek-ui.git
cd chat-deepseek-ui

Install required packages:

pip install streamlit ollama

If you still get build errors in package pyarrow, try installing pyarrow
as a binary:

pip install --only-binary=:all: pyarrow

Run the app:

streamlit run app.py

A solution for Monitoring and Logging Containers

Duc Tran — Thu, 08 Dec 2022 16:01:22 +0000

domolo

A monitoring and logging solution for Docker hosts and containers with Prometheus, Grafana, Loki, cAdvisor, NodeExporter and alerting with AlertManager.

Inspired by dockprom

Full source code in here

Install

Clone this repository on your Docker host, cd into domolo directory and run docker-compose up -d:

git clone https://github.com/ductnn/domolo.git
cd domolo
docker-compose up -d

Containers:

Prometheus (metrics database): http://<host-ip>:9090
Prometheus-Pushgateway (push acceptor for ephemeral and batch jobs): http://<host-ip>:9091
AlertManager (alerts management): http://<host-ip>:9093
Grafana (visualize metrics): http://<host-ip>:3000
Loki (likes prometheus, but for logs): http://<host-ip>:3100
Promtail (is the agent, responsible for gathering logs and sending them to Loki)
NodeExporter (host metrics collector)
cAdvisor (containers metrics collector)
Caddy (reverse proxy and basic auth provider for prometheus and alertmanager)

Grafana

Change the credentials in file config:

GF_SECURITY_ADMIN_USER=admin
GF_SECURITY_ADMIN_PASSWORD=changeme
GF_USERS_ALLOW_SIGN_UP=false

Grafana is preconfigured with dashboards, setup Prometheus(default) and Loki in datasources

apiVersion: 1

datasources:
  - name: Prometheus
    type: prometheus
    access: proxy
    orgId: 1
    url: http://prometheus:9090
    basicAuth: false
    isDefault: true
    editable: true

  - name: Loki
    type: loki
    access: proxy
    jsonData:
      maxLines: 1000
    basicAuth: false
    url: http://loki:3100
    isDefault: false
    editable: true

Prometheus + Node Exporter

Config prometheus for receiving metrics from node_exporter. First, setup node_exporter in servers we need monitor with docker-compose.agents.yml and run command:

docker-compose -f docker-compose.agents.yml up -d

This file will setup 3 agents:

node_exporter
cAdvisor
promtail

Then, we need config scrape metric on prometheus server:

Live monitoring prometheus server:

scrape_configs:
  - job_name: 'nodeexporter'
    scrape_interval: 5s
    static_configs:
      - targets: ['nodeexporter:9100']

Monitoring other Server, we need to add external_labels:

external_labels:
  monitor: 'docker-host-alpha'

scrape_configs:
  - job_name: 'ApiExporter'
    scrape_interval: 5s
    static_configs:
      - targets: ['<IP Server need Monitor>:Port']

Grafana Dashboards

Simple dashboards on Grafana:

Node Exporter

Monitor Services

Docker Host

Loki

Setup config loki in file loki-config

TODO: Setup s3

Config scrape logs with promtail, create file promtail-config.yaml and setup:

Scrape logs container:

- job_name: container_logs
  docker_sd_configs:
    - host: unix:///var/run/docker.sock
      refresh_interval: 5s
  relabel_configs:
    - source_labels: ['__meta_docker_container_name']
      regex: '/(.*)'
      target_label: 'container'

Scrape logs systems:

- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*log

Demo

Create simple tool generate logs and containerization this tool. Navigate to file entrypoint.sh and run test:

➜  domolo git:(master) cd fake-logs
➜  fake-logs git:(master) ✗ chmod +x entrypoint.sh
➜  fake-logs git:(master) ✗ ./entrypoint.sh
2022-12-08T13:20:00Z ERROR An error is usually an exception that has been caught and not handled.
2022-12-08T13:20:00Z DEBUG This is a debug log that shows a log that can be ignored.
2022-12-08T13:20:01Z WARN A warning that should be ignored is usually at this level and should be actionable.
2022-12-08T13:20:03Z ERROR An error is usually an exception that has been caught and not handled.
2022-12-08T13:20:05Z ERROR An error is usually an exception that has been caught and not handled.
2022-12-08T13:20:09Z INFO This is less important than debug log and is often used to provide context in the current task.
2022-12-08T13:20:13Z ERROR An error is usually an exception that has been caught and not handled.
2022-12-08T13:20:15Z DEBUG This is a debug log that shows a log that can be ignored.
2022-12-08T13:20:16Z INFO This is less important than debug log and is often used to provide context in the current task.
2022-12-08T13:20:17Z INFO This is less important than debug log and is often used to provide context in the current task.
...

Then, add fake-logs in docker-compose.yml

# Fake Logs
flogs:
  image: ductn4/flog:v1 # Set your name image :)
  build:
    context: ./fake-logs
    dockerfile: Dockerfile
  container_name: fake-logs
  restart: always
  networks:
    - monitor-net
  labels:
    org.label-schema.group: "monitoring"

or checkout docker-compose.with-flogs.yml and run command docker-compose -f docker-compose.with-flogs.yml up -d

Navigate grafana and open Explore:

So, we can select labels and views logs:

Ex: Select label container and view log container fake-logs:

More logs: logs system, other containers, ....

Show your support

Give a ⭐ if you like this application ❤️

Contribution

All contributions are welcomed in this project!

License

The MIT License (MIT). Please see LICENSE for more information.

Simple tool check informations from domain name with golang

Duc Tran — Tue, 20 Sep 2022 16:49:09 +0000

LOOKUP

LOOKUP is a simple tool check IP informations, CNAME, NS(Name Server), A records, MX records, TXT records with Go.

     /\/| __   /\/| __
    |/\/ /_/  |/\/ /_/
      ___     ___
     / _ \   / _ \
    | (_) | | (_) |
     \___/   \___/

Full source code in here

Fundamental

CNAME: is a Canonical Name Record or Alias Record. A type of resource record in the Domain Name System (DNS), that specifies that one domain name is an alias of another canonical domain name.
A and AAAA Record: An A and AAAA record are actually primary DNS records. They associate a domain name with a specific IP address.
TXT Record : A TXT record (short for text record) is a type of resource record in the Domain Name System (DNS) used to provide the ability to associate arbitrary text with a host or other name, such as human readable information about a server, network, data center, or other accounting information.
MX(Mail Exchanger) Record: A mail exchanger record (MX record) specifies the mail server responsible for accepting email messages on behalf of a domain name. It is a resource record in the Domain Name System (DNS). It is possible to configure several MX records, typically pointing to an array of mail servers for load balancing and redundancy.
NameServer: Nameserver is a server on the internet specialized in handling queries regarding the location of a domain name's various services. Nameservers are a fundamental part of the Domain Name System (DNS). They allow using domains instead of IP addresses.

Install

First, install golang.

Then, clone from soucre code and setup:

git clone https://github.com/ductnn/lookup.git
cd lookup
go get

Usage

Without docker

Run command:

go run main.go
# Enter domain you want to check.
# Example
google.com

Check result:

CNAME

TXT records

IP informations

Name Servers

MX records

With docker

Check Dockerfile and build with command:

docker build -t <your-image> -f Dockerfile .

or pulls my image in here, and run container

docker run -it ductn4/loo

With file binary

You can down file binary in here for running tool:

# Download file bin to your local
chmod +x loo
./loo
# And check result ...

Test with facebook.com:

➜  bin git:(master) ✗ ./loo

     /\/| __   /\/| __
    |/\/ /_/  |/\/ /_/
      ___     ___
     / _ \   / _ \
    | (_) | | (_) |
     \___/   \___/


Enter subdomain or domain name:
facebook.com

CNAME
+---------------------+--------------+---------------+
| DOMAIN OR SUBDOMAIN |    DOMAIN    |     CNAME     |
+---------------------+--------------+---------------+
| facebook.com        | facebook.com | facebook.com. |
+---------------------+--------------+---------------+

TXT records
+--------------+----------------------------------------------------------------------+
|    DOMAIN    |                             TXT RECORDS                              |
+--------------+----------------------------------------------------------------------+
| facebook.com | google-site-verification=A2WZWCNQHrGV_TWwKh6KHY90tY0SHZo_RnyMJoDaG0s |
+              +----------------------------------------------------------------------+
|              | v=spf1                                                               |
|              | redirect=_spf.facebook.com                                           |
+              +----------------------------------------------------------------------+
|              | google-site-verification=sK6uY9x7eaMoEMfn3OILqwTFYgaNp4llmguKI-C3_iA |
+              +----------------------------------------------------------------------+
|              | google-site-verification=wdH5DTJTc9AYNwVunSVFeK0hYDGUIEOGb-RReU6pJlY |
+--------------+----------------------------------------------------------------------+

IP Informations
+--------------+------------------------------------+------------------+------------------+------------------------+
|    DOMAIN    |                 IP                 | CITY AND COUNTRY |     LOCATION     |      ORGANIZATION      |
+--------------+------------------------------------+------------------+------------------+------------------------+
| facebook.com | 31.13.75.35                        | Hong             | 22.2783,114.1747 | AS32934 Facebook,      |
|              |                                    | Kong, HK         |                  | Inc.                   |
+              +------------------------------------+------------------+------------------+                        +
|              | 2a03:2880:f15a:83:face:b00c:0:25de | Dublin,          | 53.3331,-6.2489  |                        |
|              |                                    | IE               |                  |                        |
+--------------+------------------------------------+------------------+------------------+------------------------+

Name Servers
+--------------+--------------------+
|    DOMAIN    |    NAME SERVERS    |
+--------------+--------------------+
| facebook.com | b.ns.facebook.com. |
+              +--------------------+
|              | d.ns.facebook.com. |
+              +--------------------+
|              | a.ns.facebook.com. |
+              +--------------------+
|              | c.ns.facebook.com. |
+--------------+--------------------+

MX Records
+--------------+-----------------------------+
|    DOMAIN    |         MX RECORDS          |
+--------------+-----------------------------+
| facebook.com | smtpin.vvv.facebook.com.    |
|              | 10                          |
+--------------+-----------------------------+

Okkk! Thank you for reading 😍 🤩 🥳

Optimize Dockerfile images for NextJS

Duc Tran — Fri, 05 Aug 2022 03:15:00 +0000

Optimize Dockerfile images for NextJS

NextJS Docker images is too big. So, this article will focus on the way optimize dockerfile for production. Here, i will use 2 ways for optimize docker images.

Full source code in here

Setup

First, we need to create a project with nextjs. Quickly, i use an example of Vercel, project in here.

Project structure:

.
├── @types
│   └── remark-html.d.ts
├── README.md
├── _posts
│   ├── dynamic-routing.md
│   ├── hello-world.md
│   └── preview.md
├── components
│   ├── alert.tsx
│   ├── avatar.tsx
│   ├── container.tsx
│   ├── cover-image.tsx
│   ├── date-formatter.tsx
│   ├── footer.tsx
│   ├── header.tsx
│   ├── hero-post.tsx
│   ├── intro.tsx
│   ├── layout.tsx
│   ├── markdown-styles.module.css
│   ├── meta.tsx
│   ├── more-stories.tsx
│   ├── post-body.tsx
│   ├── post-header.tsx
│   ├── post-preview.tsx
│   ├── post-title.tsx
│   └── section-separator.tsx
├── interfaces
│   ├── author.ts
│   └── post.ts
├── lib
│   ├── api.ts
│   ├── constants.ts
│   └── markdownToHtml.ts
├── next-env.d.ts
├── package.json
├── pages
│   ├── _app.tsx
│   ├── _document.tsx
│   ├── index.tsx
│   └── posts
│       └── [slug].tsx
├── postcss.config.js
├── public
│   ├── assets
│   │   └── blog
│   │       ├── authors
│   │       │   ├── jj.jpeg
│   │       │   ├── joe.jpeg
│   │       │   └── tim.jpeg
│   │       ├── dynamic-routing
│   │       │   └── cover.jpg
│   │       ├── hello-world
│   │       │   └── cover.jpg
│   │       └── preview
│   │           └── cover.jpg
│   └── favicon
│       ├── android-chrome-192x192.png
│       ├── android-chrome-512x512.png
│       ├── apple-touch-icon.png
│       ├── browserconfig.xml
│       ├── favicon-16x16.png
│       ├── favicon-32x32.png
│       ├── favicon.ico
│       ├── mstile-150x150.png
│       ├── safari-pinned-tab.svg
│       └── site.webmanifest
├── styles
│   └── index.css
├── next.config.js
├── tailwind.config.js
└── tsconfig.json

Then, install and build this blog:

➜  blog-starter git:(master) ✗ yarn
➜  blog-starter git:(master) ✗ yarn build
➜  blog-starter git:(master) ✗ yarn start

Your blog should be up and running on localhost:3000.

Build on Docker

Ignore unneeded files with .dockerignore:

node_modules
*.DS_Store
.next
.gitignore
README.md
.dockerignore
LICENSE
.docker
.gitlab

We have 3 scenarios dockerfile in here. First, i'll use Basic Dockerfile:

➜  blog-starter git:(master) ✗ docker build -t blog-with-basic-dockerfile -f .docker/basic.dockerfile .

# Check docker images
➜  blog-starter git:(master) ✗ docker images
REPOSITORY                                       TAG                               IMAGE ID       CREATED          SIZE
blog-with-basic-dockerfile                       latest                            b70f75178890   8 seconds ago    370MB

In this scenarios, we built nextjs image with size 370MB.

Next, we use multi-stage docker with Multi Stage Docker:

➜  blog-starter git:(master) ✗ docker build -t blog-with-multistage-dockerfile -f .docker/multistage.dockerfile .

# Check docker images
➜  blog-starter git:(master) ✗ docker images
REPOSITORY                                           TAG                               IMAGE ID       CREATED           SIZE
blog-with-multistage-dockerfile                      latest                            07c84ea2173a   38 seconds ago    339MB

In this way, i use module node-prune in stage BUILD after install packages. node-prune can remove unnecessary files from node_modules:

         files total 43,924
       files removed 12,814
        size removed 28 MB
            duration 866ms

Hehe, reduced 28MB. You can follow this guide for setup in your base node image. So, this images has been optimized 370MB -> 339MB.

Finally, we build dockerfile with multi-stage docker and enable mode standalone of NextJS. Create file next.config.js with content:

module.exports = {
    output: "standalone"
}

Nextjs can automatically create a standalone folder which copies only the necessary files for a production deployment including select files in node_modules. More docs.

Let's start with Dockerfile:

➜  blog-starter git:(master) ✗ docker build -t blog-with-multistage-standalone-dockerfile -f .docker/multistage_standalone.dockerfile .

# Check docker images
➜  blog-starter git:(master) ✗ docker images
REPOSITORY                                                      TAG                     IMAGE ID       CREATED           SIZE
blog-with-multistage-standalone-dockerfile                      latest                  07c84ea2173a   38 seconds ago    119MB

WoW! Great ... The size of images is only 119MB.

So, I have presented 3 ways to optimize image size but ... it build very slow. In the next post, i will show you how to build image faster.

Thanks for reading 😁😁😁

Simple tool crawl urls form domain

Duc Tran — Sat, 30 Jul 2022 11:06:00 +0000

cUrls is a simple tool crawl urls from domain using colly
library. Source code in here.

Installation

First, install golang

Then, clone from soure code and install:

git clone https://github.com/ductnn/cUrls.git
cd cUrls
go get

Usage

Run command:

go run curls.go > sub.txt
# Enter domain you want to crawl.
# Example
http://httpbin.org/

Check results in file sub.txt:

Visiting http://httpbin.org/
Link found: "\n        \n            \n            \n            \n        \n    " -> https://github.com/requests/httpbin
Link found: "the developer - Website" -> https://kennethreitz.org
Link found: "Send email to the developer" -> mailto:me@kennethreitz.org
Link found: "Flasgger" -> https://github.com/rochacbruno/flasgger
Link found: "HTML form" -> /forms/post

So done !!! =))))

Show your support

Give a ⭐ if you like this application ❤️

Contribution

Contributions are more than welcome in this project!

License

The MIT License (MIT). Please see LICENSE for more information.

Build API Server for upload files to AWS S3

Duc Tran — Tue, 29 Mar 2022 16:32:19 +0000

Build API Server for upload files to AWS S3

Full source code in here

Project structure

Overview:

.
├── ./src
├── ./Dockerfile
├── ./README.md
├── ./package-lock.json
├── ./package.json
└── ./tsconfig.json

Detail:

src
├── config
│   ├── config.ts
│   └── index.ts
├── main.ts
├── routes
│   ├── index.ts
│   └── upload-to-S3.ts
├── services
│   ├── api.ts
│   └── index.ts
└── utils
    ├── index.ts
    └── utils.ts

Install and build

With manual

Clone this repo:

git clone https://github.com/ductnn/api-upload-to-s3.git
cd api-upload-to-s3

Set the enviroment variables:

cp .env.example .env

# open .env and modify the environment variables
## AWS_ACCESS_KEY=
## AWS_SECRET_KEY=
## AWS_REGION_DEFAULT=
## AWS_BUCKET_NAME=

Install the dependencies:

npm install

Start API Server woth command npm start:

➜  api-upload-to-s3 git:(master) npm start 

> api-upload-to-s3@1.0.0 start
> node --require ts-node/register src/main.ts

Running service:  undefined
🚀 Server started as undefined at http://localhost:5000

Open url 127.0.0.1:5000 on browser to view result or use curl:

➜  api-upload-to-s3 git:(master) ✗ curl 127.0.0.1:5000
<h3>Build API Server for upload files to AWS S3</h3>

Then, we use Postman with method PUT in 127.0.0.1:5000/s3/upload to
upload files.

With Dockerfile

Easy to build API with Dockerfile:

# Build images
docker build -t <YOUR-DOCKER-ID>/api-upload-to-s3:v1 -f Dockerfile .

# Push to your registry
docker push <YOUR-DOCKER-ID>/api-upload-to-s3:v1

# And run this docker image
docker run -itd -p 5000:5000 <YOUR-DOCKER-ID>/api-upload-to-s3:v1

Contribution

Contributions are more than welcome in this project!

License

The MIT License (MIT). Please see LICENSE for more information.

MongoDB cluster on Docker

Duc Tran — Fri, 28 Jan 2022 17:40:26 +0000

## MongoDB Cluster on Docker

A simple implement MongoDB Cluster on Docker with PSS Style (Primary - Secondary - Secondary). Inspired by this article

Source code in here

Components

2 Mongos (router): The mongos acts as query routers, providing an interface between client applications and the sharded cluster.
- router01
- router02
3 Config Servers: Store metadata and configuration settings for the cluster
- configsvr01
- configsvr02
- configsvr03
3 Shards (each a 3 member replica set): Each shard contains a subset of the sharded data. Each shard can be deployed as a replica set.
- shard01-a, shard01-b, shard01-c
- shard02-a, shard02-b, shard02-c
- shard03-a, shard03-b, shard03-c

Installation

Clone this repo:

git clone https://github.com/ductnn/mongodb-cluster-docker.git
cd mongodb-cluster-docker

And run docker-compose.yaml:

➜  mongodb-cluster-docker git:(master) ✗ docker-compose up -d
Creating network "mongodb-cluster-docker_default" with the default driver
Pulling router01 (mongo:5.0.3)...
5.0.3: Pulling from library/mongo
7b1a6ab2e44d: Already exists
90eb44ebc60b: Already exists
5085b59f2efb: Already exists
c7499923d022: Already exists
019496b6c44a: Already exists
c0df4f407f69: Already exists
351daa315b6c: Already exists
557b07ecd9d7: Already exists
a2dff157a5e3: Already exists
07d83e88231b: Already exists
Digest: sha256:07212fb304ea36b8c5a9e5694527f16deeb0b99f87fc60162dc15ab260bf8a2a
Status: Downloaded newer image for mongo:5.0.3
Creating shard-02-node-b ... done
Creating shard-03-node-b ... done
Creating shard-03-node-c ... done
Creating shard-02-node-c ... done
Creating router-01       ... done
Creating shard-01-node-b ... done
Creating shard-01-node-c ... done
Creating shard-03-node-a ... done
Creating shard-01-node-a ... done
Creating shard-02-node-a ... done
Creating router-02       ... done
Creating mongo-config-01 ... done
Creating mongo-config-02 ... done
Creating mongo-config-03 ... done

Result:

➜  mongodb-cluster-docker git:(master) ✗ docker ps
CONTAINER ID   IMAGE         COMMAND                  CREATED          STATUS          PORTS                                           NAMES
302243ef45a3   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 15 minutes   0.0.0.0:27121->27017/tcp, :::27121->27017/tcp   mongo-config-03
4599103daf74   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27120->27017/tcp, :::27120->27017/tcp   mongo-config-02
5be686736dd8   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27119->27017/tcp, :::27119->27017/tcp   mongo-config-01
4eff0c607d3c   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27118->27017/tcp, :::27118->27017/tcp   router-02
b28d39a0a262   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27125->27017/tcp, :::27125->27017/tcp   shard-02-node-a
ad9c4985f630   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27122->27017/tcp, :::27122->27017/tcp   shard-01-node-a
5a43e22ff9ae   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27128->27017/tcp, :::27128->27017/tcp   shard-03-node-a
52520b1b29a9   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27124->27017/tcp, :::27124->27017/tcp   shard-01-node-c
9591f85f2dc0   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27130->27017/tcp, :::27130->27017/tcp   shard-03-node-c
1b745802b88a   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27126->27017/tcp, :::27126->27017/tcp   shard-02-node-b
161baac14972   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27117->27017/tcp, :::27117->27017/tcp   router-01
4468e7f23da1   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27123->27017/tcp, :::27123->27017/tcp   shard-01-node-b
1b99d3d5ea54   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27129->27017/tcp, :::27129->27017/tcp   shard-03-node-b
0480f8b2247d   mongo:5.0.3   "docker-entrypoint.s…"   16 minutes ago   Up 16 minutes   0.0.0.0:27127->27017/tcp, :::27127->27017/tcp   shard-02-node-c

Then, we need to init config-servers, shards and router:

Config servers

# Initilizing the config server
➜  mongodb-cluster-docker git:(master) ✗ bash bash/init-configserver.sh 
MongoDB shell version v5.0.3
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("555e23b7-f1bd-4232-9947-8f5dd0db6c3b") }
MongoDB server version: 5.0.3
================
Warning: the "mongo" shell has been superseded by "mongosh",
which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed in
an upcoming release.
We recommend you begin using "mongosh".
For installation instructions, see
https://docs.mongodb.com/mongodb-shell/install/
================
{
        "ok" : 1,
        "$gleStats" : {
                "lastOpTime" : Timestamp(1636305337, 1),
                "electionId" : ObjectId("000000000000000000000000")
        },
        "lastCommittedOpTime" : Timestamp(0, 0)
}
bye

Shards

# Initilzing the shards
➜  mongodb-cluster-docker git:(master) ✗ bash bash/init-replicaset.sh 
MongoDB shell version v5.0.3
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("815f9a5c-997b-48a6-bba0-8847b0890521") }
MongoDB server version: 5.0.3
================
Warning: the "mongo" shell has been superseded by "mongosh",
which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed in
an upcoming release.
We recommend you begin using "mongosh".
For installation instructions, see
https://docs.mongodb.com/mongodb-shell/install/
================
{
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1636305394, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1636305394, 1)
}
bye
MongoDB shell version v5.0.3
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("16424801-e037-450b-b4db-4d9dfe19752b") }
MongoDB server version: 5.0.3
================
Warning: the "mongo" shell has been superseded by "mongosh",
which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed in
an upcoming release.
We recommend you begin using "mongosh".
For installation instructions, see
https://docs.mongodb.com/mongodb-shell/install/
================
{ "ok" : 1 }
bye
MongoDB shell version v5.0.3
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("6f621971-1d81-45c6-9f74-23eacc9cc4a1") }
MongoDB server version: 5.0.3
================
Warning: the "mongo" shell has been superseded by "mongosh",
which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed in
an upcoming release.
We recommend you begin using "mongosh".
For installation instructions, see
https://docs.mongodb.com/mongodb-shell/install/
================
{
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1636305395, 1),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1636305395, 1)
}
bye

Routers

After initilizing config-server and shards, wait a bit to to elect primaries
before initilizing router:

➜  mongodb-cluster-docker git:(master) ✗ bash bash/init-router.sh 
MongoDB shell version v5.0.3
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("25a1c2a2-135c-4550-b732-713e018eb0a2") }
MongoDB server version: 5.0.3
================
Warning: the "mongo" shell has been superseded by "mongosh",
which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed in
an upcoming release.
We recommend you begin using "mongosh".
For installation instructions, see
https://docs.mongodb.com/mongodb-shell/install/
================
{
        "shardAdded" : "rs-shard-01",
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1636305656, 6),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1636305656, 5)
}
{
        "shardAdded" : "rs-shard-02",
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1636305660, 2),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1636305658, 4)
}
{
        "shardAdded" : "rs-shard-03",
        "ok" : 1,
        "$clusterTime" : {
                "clusterTime" : Timestamp(1636305662, 9),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1636305662, 3)
}
bye

Verify status

We can check the status of replicasets:

# Exec to the shard
➜  mongodb-cluster-docker git:(master) ✗ docker exec -it shard-01-node-a bash 
root@ad9c4985f630:/# mongo
rs-shard-01:PRIMARY> rs.status()
{
        "set" : "rs-shard-01",
        "date" : ISODate("2021-11-07T17:31:01.326Z"),
        "myState" : 1,
        "term" : NumberLong(1),
        "syncSourceHost" : "",
        "syncSourceId" : -1,
        "heartbeatIntervalMillis" : NumberLong(2000),
        "majorityVoteCount" : 2,
        "writeMajorityCount" : 2,
        "votingMembersCount" : 3,
        "writableVotingMembersCount" : 3,
        "optimes" : {
                "lastCommittedOpTime" : {
                        "ts" : Timestamp(1636306261, 9),
                        "t" : NumberLong(1)
                },
                "lastCommittedWallTime" : ISODate("2021-11-07T17:31:01.283Z"),
                "readConcernMajorityOpTime" : {
                        "ts" : Timestamp(1636306261, 9),
                        "t" : NumberLong(1)
                },
                "appliedOpTime" : {
                        "ts" : Timestamp(1636306261, 9),
                        "t" : NumberLong(1)
                },
                "durableOpTime" : {
                        "ts" : Timestamp(1636306261, 8),
                        "t" : NumberLong(1)
                },
                "lastAppliedWallTime" : ISODate("2021-11-07T17:31:01.283Z"),
                "lastDurableWallTime" : ISODate("2021-11-07T17:31:01.278Z")
        },
        "lastStableRecoveryTimestamp" : Timestamp(1636306245, 14),
        "electionCandidateMetrics" : {
                "lastElectionReason" : "electionTimeout",
                "lastElectionDate" : ISODate("2021-11-07T17:16:45.266Z"),
                "electionTerm" : NumberLong(1),
                "lastCommittedOpTimeAtElection" : {
                        "ts" : Timestamp(0, 0),
                        "t" : NumberLong(-1)
                },
                "lastSeenOpTimeAtElection" : {
                        "ts" : Timestamp(1636305394, 1),
                        "t" : NumberLong(-1)
                },
                "numVotesNeeded" : 2,
                "priorityAtElection" : 1,
                "electionTimeoutMillis" : NumberLong(10000),
                "numCatchUpOps" : NumberLong(0),
                "newTermStartDate" : ISODate("2021-11-07T17:16:45.299Z"),
                "wMajorityWriteAvailabilityDate" : ISODate("2021-11-07T17:16:45.957Z")
        },
        "members" : [
                {
                        "_id" : 0,
                        "name" : "shard01-a:27017",
                        "health" : 1,
                        "state" : 1,
                        "stateStr" : "PRIMARY",
                        "uptime" : 1211,
                        "optime" : {
                                "ts" : Timestamp(1636306261, 9),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2021-11-07T17:31:01Z"),
                        "syncSourceHost" : "",
                        "syncSourceId" : -1,
                        "infoMessage" : "",
                        "electionTime" : Timestamp(1636305405, 1),
                        "electionDate" : ISODate("2021-11-07T17:16:45Z"),
                        "configVersion" : 2,
                        "configTerm" : 1,
                        "self" : true,
                        "lastHeartbeatMessage" : ""
                },
                {
                        "_id" : 1,
                        "name" : "shard01-b:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 867,
                        "optime" : {
                                "ts" : Timestamp(1636306260, 9),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1636306260, 9),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2021-11-07T17:31:00Z"),
                        "optimeDurableDate" : ISODate("2021-11-07T17:31:00Z"),
                        "lastHeartbeat" : ISODate("2021-11-07T17:31:01.263Z"),
                        "lastHeartbeatRecv" : ISODate("2021-11-07T17:31:01.252Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "shard01-a:27017",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 2,
                        "configTerm" : 1
                },
                {
                        "_id" : 2,
                        "name" : "shard01-c:27017",
                        "health" : 1,
                        "state" : 2,
                        "stateStr" : "SECONDARY",
                        "uptime" : 867,
                        "optime" : {
                                "ts" : Timestamp(1636306260, 9),
                                "t" : NumberLong(1)
                        },
                        "optimeDurable" : {
                                "ts" : Timestamp(1636306260, 9),
                                "t" : NumberLong(1)
                        },
                        "optimeDate" : ISODate("2021-11-07T17:31:00Z"),
                        "optimeDurableDate" : ISODate("2021-11-07T17:31:00Z"),
                        "lastHeartbeat" : ISODate("2021-11-07T17:31:01.263Z"),
                        "lastHeartbeatRecv" : ISODate("2021-11-07T17:31:01.248Z"),
                        "pingMs" : NumberLong(0),
                        "lastHeartbeatMessage" : "",
                        "syncSourceHost" : "shard01-a:27017",
                        "syncSourceId" : 0,
                        "infoMessage" : "",
                        "configVersion" : 2,
                        "configTerm" : 1
                }
        ],
        "ok" : 1,
        "$gleStats" : {
                "lastOpTime" : Timestamp(0, 0),
                "electionId" : ObjectId("7fffffff0000000000000001")
        },
        "lastCommittedOpTime" : Timestamp(1636306261, 9),
        "$configServerState" : {
                "opTime" : {
                        "ts" : Timestamp(1636306261, 11),
                        "t" : NumberLong(-1)
                }
        },
        "$clusterTime" : {
                "clusterTime" : Timestamp(1636306261, 11),
                "signature" : {
                        "hash" : BinData(0,"AAAAAAAAAAAAAAAAAAAAAAAAAAA="),
                        "keyId" : NumberLong(0)
                }
        },
        "operationTime" : Timestamp(1636306261, 9)
}

Similar with another shards. Check the status of router:

➜  mongodb-cluster-docker git:(master) ✗ docker-compose exec router01 mongo --port 27017
MongoDB shell version v5.0.3
connecting to: mongodb://127.0.0.1:27017/?compressors=disabled&gssapiServiceName=mongodb
Implicit session: session { "id" : UUID("69408b5a-d313-44ae-ae64-81e9d3d21588") }
MongoDB server version: 5.0.3
================
Warning: the "mongo" shell has been superseded by "mongosh",
which delivers improved usability and compatibility.The "mongo" shell has been deprecated and will be removed in
an upcoming release.
We recommend you begin using "mongosh".
For installation instructions, see
https://docs.mongodb.com/mongodb-shell/install/
================
---
The server generated these startup warnings when booting: 
        2021-11-07T17:10:47.048+00:00: Access control is not enabled for the database. Read and write access to data and configuration is unrestricted
---
mongos> sh.status()
--- Sharding Status --- 
  sharding version: {
        "_id" : 1,
        "minCompatibleVersion" : 5,
        "currentVersion" : 6,
        "clusterId" : ObjectId("618809c41517a9e726602e13")
  }
  shards:
        {  "_id" : "rs-shard-01",  "host" : "rs-shard-01/shard01-a:27017,shard01-b:27017,shard01-c:27017",  "state" : 1,  "topologyTime" : Timestamp(1636305656, 2) }
        {  "_id" : "rs-shard-02",  "host" : "rs-shard-02/shard02-a:27017,shard02-b:27017,shard02-c:27017",  "state" : 1,  "topologyTime" : Timestamp(1636305658, 2) }
        {  "_id" : "rs-shard-03",  "host" : "rs-shard-03/shard03-a:27017,shard03-b:27017,shard03-c:27017",  "state" : 1,  "topologyTime" : Timestamp(1636305662, 1) }
  active mongoses:
        "5.0.3" : 2
  autosplit:
        Currently enabled: yes
  balancer:
        Currently enabled: yes
        Currently running: yes
        Collections with active migrations: 
                config.system.sessions started at Sun Nov 07 2021 17:33:54 GMT+0000 (UTC)
        Failed balancer rounds in last 5 attempts: 0
        Migration results for the last 24 hours: 
                237 : Success
  databases:
        {  "_id" : "config",  "primary" : "config",  "partitioned" : true }
                config.system.sessions
                        shard key: { "_id" : 1 }
                        unique: false
                        balancing: true
                        chunks:
                                rs-shard-01     787
                                rs-shard-02     118
                                rs-shard-03     119
                        too many chunks to print, use verbose if you want to force print

Done ... wait. You can remove all:

➜  mongodb-cluster-docker git:(master) bash clean_all.sh 
Stopping mongo-config-03 ... done
Stopping mongo-config-02 ... done
Stopping mongo-config-01 ... done
Stopping router-02       ... done
Stopping shard-02-node-a ... done
Stopping shard-01-node-a ... done
Stopping shard-03-node-a ... done
Stopping shard-01-node-c ... done
Stopping shard-03-node-c ... done
Stopping shard-02-node-b ... done
Stopping router-01       ... done
Stopping shard-01-node-b ... done
Stopping shard-03-node-b ... done
Stopping shard-02-node-c ... done
Removing mongo-config-03 ... done
Removing mongo-config-02 ... done
Removing mongo-config-01 ... done
Removing router-02       ... done
Removing shard-02-node-a ... done
Removing shard-01-node-a ... done
Removing shard-03-node-a ... done
Removing shard-01-node-c ... done
Removing shard-03-node-c ... done
Removing shard-02-node-b ... done
Removing router-01       ... done
Removing shard-01-node-b ... done
Removing shard-03-node-b ... done
Removing shard-02-node-c ... done
Removing network mongodb-cluster-docker_default
Removing image mongo:5.0.3

And so real done =))))

Show your support

Give a ⭐ if you like this application ❤️

Contribution

All contributions are welcomed in this project!

License

The MIT License (MIT). Please see LICENSE for more information.

Install Jenkins with Docker 🐳

Duc Tran — Tue, 30 Nov 2021 17:41:21 +0000

Downloading and running Jenkins on Docker

The recommended Docker image to use is the Official jenkins/jenkins
image from Docker hub repo. A new jenkins/jenkins
image is published each time a new release of Jenkins Docker is published (Don't use Jenkins image - Already deprecated).
OK!!! Let's start.

Note: To install Jenkins, we need to deploy 2 containers: docker:dind and jenkins/jenkins.

Step 1: Create bridge network for 2 container

docker network create jenkins

Step 2: Setup docker:dind (DinD: Docker-in-Docker): is just what it says running Docker inside a Docker container

docker run --name jenkins-docker -d \
  --privileged --network jenkins --network-alias docker \
  --env DOCKER_TLS_CERTDIR=/certs \
  --volume ~/docker-certs-jk:/certs/client \
  --volume ~/jenkins-home:/var/jenkins_home \
  -p 2376:2376 docker:dind --storage-driver overlay2

--privileged: Running Docker in Docker currently requires privileged access to function properly.

--network-alias docker: Makes the Docker in Docker container available as the hostname docker within the jenkins network.

--env DOCKER_TLS_CERTDIR=/certs: Enables the use of TLS in the Docker server.

--volume ~/jenkins-home:/var/jenkins_home: Mounting volume jenkins-home to workspace directory of Jenkins.

Steps 3: Deploy container Jenkins. We can use command to run container:

docker run --name jenkins -d \
  --network jenkins --env DOCKER_HOST=tcp://docker:2376 \
  --env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 \
  -p 8080:8080 -p 50000:50000 \
  -v ~/jenkins_home:/var/jenkins_home \
  -v ~/docker-certs-jk:/certs/client:ro \
  jenkins/jenkins:2.289.3-lts-jdk11

But i'll custom for my new jenkins image. Create Dockerfile:

FROM jenkins/jenkins:2.289.3-lts-jdk11

USER root

RUN apt-get update && apt-get install -y apt-transport-https \
    ca-certificates curl gnupg2 \
    software-properties-common
RUN curl -fsSL https://download.docker.com/linux/debian/gpg | apt-key add -
RUN apt-key fingerprint 0EBFCD88
RUN add-apt-repository \
     "deb [arch=amd64] https://download.docker.com/linux/debian \
     $(lsb_release -cs) stable"
RUN apt-get update && apt-get install -y docker-ce-cli

USER jenkins

Note: You can see more in my repo
Navigate to directory with Dockerfile and build new image:

docker build -t jenkins-new:v1 .

Then, we run the new jenkins image as container in Docker:

docker run --name jenkins -d \
  --network jenkins --env DOCKER_HOST=tcp://docker:2376 \
  --env DOCKER_CERT_PATH=/certs/client --env DOCKER_TLS_VERIFY=1 \
  -p 8080:8080 -p 50000:50000 \
  -v ~/jenkins_home:/var/jenkins_home \
  -v ~/docker-certs-jk:/certs/client:ro \
  jenkins-new:v1

Check our result:

➜  jenkins git:(main) ✗ docker ps
CONTAINER ID   IMAGE            COMMAND                  CREATED        STATUS         PORTS                                                                                      NAMES
d5609c0f9f69   jenkins-new:v1   "/sbin/tini -- /usr/…"   18 hours ago   Up 9 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:50000->50000/tcp, :::50000->50000/tcp   jenkins
339877c8ca20   docker:dind      "dockerd-entrypoint.…"   18 hours ago   Up 5 seconds   2375/tcp, 0.0.0.0:2376->2376/tcp, :::2376->2376/tcp                                        jenkins-docker

Now, we need to config Jenkins after installation, let access the dashboard at localhost:8080, and unlock jenkins using password:

Follow the tutorial, we can take the password at /var/lib/jenkins/secrets/initialAdminPassword

CONTAINER ID   IMAGE            COMMAND                  CREATED        STATUS          PORTS                                                                                      NAMES
d5609c0f9f69   jenkins-new:v1   "/sbin/tini -- /usr/…"   19 hours ago   Up 29 minutes   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:50000->50000/tcp, :::50000->50000/tcp   jenkins
339877c8ca20   docker:dind      "dockerd-entrypoint.…"   19 hours ago   Up 29 minutes   2375/tcp, 0.0.0.0:2376->2376/tcp, :::2376->2376/tcp                                        jenkins-docker
➜  jenkins git:(main) ✗ docker exec -it d5 bash
jenkins@d5609c0f9f69:/$ cat /var/lib/jenkins/secrets/initialAdminPassword
YOUR_PASSWORD_IN_HERE

After unlock jenkins, the Customize Jenkins page appears. We choose Install suggested plugins, and Create Admin User. Jenkins ready to use 🤟 🤟 🤟.

And we can install BlueOcean plugins (GUI for CI/CD Pipeline)

OK Done !!! Thanks you for reading my article 🥰🥰🥰

Shopping on my store 🥰🥰🥰

DEV Community: Duc Tran

Chat UI for Deepseek in your local

Chat deepseek UI

Features

Setup

A solution for Monitoring and Logging Containers

domolo

Install

Grafana

Prometheus + Node Exporter

Grafana Dashboards

Loki

Demo

Show your support

Contribution

License

Simple tool check informations from domain name with golang

LOOKUP

Fundamental

Install

Usage

Without docker

With docker

With file binary

Optimize Dockerfile images for NextJS

Optimize Dockerfile images for NextJS

Setup

Build on Docker

Simple tool crawl urls form domain

Installation

Usage

Show your support

Contribution

License

Build API Server for upload files to AWS S3

Build API Server for upload files to AWS S3

Project structure

Install and build

With manual

With Dockerfile

Contribution

License

MongoDB cluster on Docker

Components

Installation

Config servers

Shards

Routers

Verify status

Show your support

Contribution

License

Install Jenkins with Docker 🐳

Downloading and running Jenkins on Docker

References