DEV Community: Jinna Balu

Deploy libreoffice as container

Jinna Balu — Fri, 27 Sep 2024 07:11:10 +0000

Configuration saved in github repo infinite-docker-compose

Create the docker-compose.yml for the obsidian

services:
  libreoffice:
    image: lscr.io/linuxserver/libreoffice:latest
    container_name: libreoffice
    security_opt:
      - seccomp=unconfined
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
    ports:
      - "3000:3000"
      - "3001:3001"
    restart: unless-stopped

Run the application docker-compose up -d

Application is available on http://localhost:3000/

Deploy obsidian as container

Jinna Balu — Fri, 27 Sep 2024 06:32:13 +0000

https://obsidian.md/ note taking app
Configuration saved in github repo infinite-docker-compose

Create the docker-compose.yml for the obsidian

services:
  obsidian:
    image: lscr.io/linuxserver/obsidian:latest
    container_name: obsidian
    ports:
      - "3000:3000"
      - "3001:3001"
    shm_size: "1gb"
    restart: unless-stopped

Run the application docker-compose up -d

Application is available on http://localhost:3000/

Deploy firefox as container

Jinna Balu — Fri, 27 Sep 2024 06:03:54 +0000

Firefox + Caddy + HTTPS for firefox.jinnabalu.local

Create the folder and create the following files, here is the tree, configuration is available in github repo infinite-docker-compose

firefox tree
.
├── Caddyfile
└── docker-compose.yml

1 directory, 2 files

docker-compose.yml

services:
  firefox:
    image: lscr.io/linuxserver/firefox:latest
    container_name: firefox
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Etc/UTC
    ports:
      - "3001:3001"
    shm_size: "1gb"
    restart: unless-stopped

  caddy:
    image: caddy:latest
    container_name: caddy
    depends_on:
      - firefox
    ports:
      - "80:80"
      - "443:443"
    volumes:
      - ./Caddyfile:/etc/caddy/Caddyfile
      - caddy_data:/data
      - caddy_config:/config
    restart: unless-stopped

volumes:
  caddy_data:
  caddy_config:

Caddyfile

firefox.jinnabalu.local {
    reverse_proxy firefox:3001
    tls internal
}

Update the host for mapping the domain, this is not required if you are mapping to the DNS records from the internet registry.

sudo nano /etc/hosts

Add the following record

127.0.0.1 firefox.jinnabalu.local

Finally run the application docker compose up -d, open https://firefox.jinnabalu.local

Identify the Active Controller In Kafka

Jinna Balu — Sun, 20 Mar 2022 16:08:03 +0000

What is Controller?

The Controller is one of the kafka broker's, usually behaves as a normal broker and having a controlling capability of managing partition state and replicas and reassignment of the partition.

How to get the current active controller?

Zookeeper is the storage of the kafka cluster, who elects the controller node either in the beginning of the cluster setup or when the controller node crashes. Here is the command to get the controller

Shell command when installed on the host

zookeeper-shell 192.168.1.1:22181 get /controller

When we are dealing with the docker container we can execute the above command in interactive mode as follows

# Connect to Zookeeper
docker run --net=host --rm confluentinc/cp-kafka:4.0.0 zookeeper-shell 192.168.1.1:22181

# Get the Controller
docker run --net=host --rm confluentinc/cp-kafka:4.0.0 zookeeper-shell 192.168.1.1:22181 get /controller

Note: 192.168.1.1 is the zookeeper IP

Elasticsearch as a container for beginners

Jinna Balu — Fri, 29 May 2020 02:55:20 +0000

Elasticsearch Single Node with Kibana

In this scenario, you'll learn how to deploy a Elasticsearch and Kibana as a Docker Container.

Elasticsearch single is recommended for dev and monitoring but not for the production or primary store.

If we have a proper backup strategy still we can risk running a single node in the production too but we don't get all the feature set like fault tolerance and distributed cluster, this is at high risk.

Let us understand how we can start the single node container in this scenario, expecting that you are familiar with docker and docker-compose commands.

Agenda

Create the docker-compose
Understand the properties used in Elasticsearch and Kibana
Health check with
- Kibana Console UI
- curl Command
Create a simple document with
- mappings
- settings
Insert Data into the created index

Create the docker-compose

Create the docker-compose.yml with the following

version: "3"
services:
    elasticsearch:
        image: docker.elastic.co/elasticsearch/elasticsearch:7.7.0
        container_name: elasticsearch
        environment:
            - discovery.type=single-node
            - ES_JAVA_OPTS="-Xms1g -Xmx1g"
        volumes:
            - vibhuviesdata:/usr/share/elasticsearch/data
        ports:
            - 9200:9200
        networks:
            - elastic
        labels:
            - co.elastic.logs/module=elasticsearch
            - co.elastic.metrics/module=elasticsearch
    kibana:
        image: docker.elastic.co/kibana/kibana:7.7.0
        container_name: kibana
        ports:
            - 5601:5601
        depends_on:
            - elasticsearch
        environment:
            ELASTICSEARCH_URL: http://elasticsearch:9200
            ELASTICSEARCH_HOSTS: http://elasticsearch:9200
        networks:
            - elastic
networks:
    elastic:
      driver: bridge  
volumes:
    vibhuviesdata:
      driver: local

Check the content of docker-compose.yml file

cat docker-compose.yml

Understands the Environment and Run the container

As mentioned in the intro hope you are familiar with the docker container commands, expecting that we proceed with the environment we have configured for the elasticsearch.

In the docker-compose we have defined two services elasticsearch and kibana. Let's look at the other properties defined.

Elasticsearch

image: docker.elastic.co/elasticsearch/elasticsearch:7.7.0 - Image of version 7.7.0
container_name - Custom name of the container
environment - To run the single need we need to set the container environment with discovery.type: single-node and Optional but important when we are running in dev machine ES_JAVA_OPTS: "-Xms512m -Xmx1024m"
volumes to maintain the persistancy on restarting the container else we loose the data when we restart the container. Here we have host volume vibhuviesdata and container volume /usr/share/elasticsearch/data. Host Volume maintains the data for multiple restarts of the container.
ports - Elasticsearch will use port 9200 for requests and port 9300 for communication between nodes within the cluster.
networks - To maintain the security from the other networks in the docker we have created a common network for both kibana and elasticsearch

Kibana

image: docker.elastic.co/kibana/kibana:7.7.0 - We have to use the same version as elasticsearch after 4.6.*. Check the Kibana Compatibility with Elasticsearch Matrix
container_name - Custom name of the container
environment - Kibana is connecting to the container elasticsearch with teh service name and port.
ports - Kibana will use port 9200 for visualising the elasticsearch
depends_on - The property tell the Kibana service to run after elasticsearch

Run the container

docker-compose up -d

Docker Commands

Check the status of the container

docker container ls -a

Check the logs of the Elasticsearch

docker logs elasticsearch

Check the logs of the Kibana

docker logs kibana

Output contains
- Elasticsearch - "message": "Cluster health status changed from [YELLOW] to [GREEN]
- Kibana - "message":"http server running at http://0:5601"

This means that both containers are running successfully

Interaction with Elasticsearch Cluster API

We have Elasticsearch running, let's try to interact with elaticsearch APIs

We have two options to intearct with Elasticsearch curl and Kibana Console UI

Curl

Cluster health API

curl -X GET "localhost:9200/_cluster/health?pretty"

Cluster state API

curl -X GET "localhost:9200/_cluster/state/_all?pretty"

Cluster Stats API

curl -X GET "localhost:9200/_cluster/stats"

Cluster Settings API

curl -X GET "localhost:9200/_cluster/settings"

Kibana UI

Cluster health API - GET /_cluster/health
Cluster state API - GET /_cluster/state/_all
Cluster Stats API - GET /_cluster/stats
Cluster Settings API - GET /_cluster/settings

Frequently used commands

Check the status of the Elasticsearch
Check the health status of the Elasticsearch

curl -X GET "localhost:9200/_cat/health"

Get the number of nodes of the Elasticsearch Cluster

curl -X GET "localhost:9200/_cat/nodes"

Check with the shards with

curl -X GET "localhost:9200/_cat/shards"

Get list of indices

curl -X GET "localhost:9200/_cat/indices?v"

Get list of indices with specific column, we want to the column index, which will list the index names

curl -X GET "localhost:9200/_cat/indices?v&h=index"

Get the list of indices sort by column

curl -X GET "localhost:9200/_cat/indices?v&s=docs.count:desc"

curl -X GET "localhost:9200/_cat/indices?v&s=docs.count:asc"

curl -X GET "localhost:9200/_cat/indices?v&s=index"

curl -X GET "localhost:9200/_cat/indices?v&s=docs.count:desc"

curl -X GET "localhost:9200/_cat/indices?v&s=docs.count:desc"

Create Index

Let's get into the actual dev stuff required for any developer to use elasticsearch with any client. Lets create a index using Index API.

Create index

curl -X PUT "localhost:9200/twitter?pretty"

Create the index with settings

curl -sXPUT 'http://localhost:9200/customer/?pretty' -d '{
  "settings" : {
      "index" : {
          "number_of_shards" : 5,
          "number_of_replicas" : 0
      }
  }
}'

Insert bulk data into the index created

for i in `seq 1 500`; do
  curl -sXPUT "localhost:9200/customer/external/$i?pretty" -d "
  {
    \"number\": $i,
    \"name\": \"John Doe - $i\"
  }"
done

Create student entity with 1000 records

curl -sXPUT 'http://localhost:9200/student/?pretty' -d '{
  "settings" : {
      "index" : {
          "number_of_shards" : 5,
          "number_of_replicas" : 0
      }
  }
}'

for i in `seq 1 20`; do
  curl -sXPUT "localhost:9200/student/external/$i?pretty" -d "
  {
    \"number\": $i,
    \"name\": \"Ram - $i\"
  }"
done

Create index with mappings and settings

curl -X PUT "localhost:9200/school?pretty" -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "number_of_shards" : 3,
        "number_of_replicas" : 2
    },
    "mappings" : {
        "properties" : {
            "name" : { "type" : "text" }
        }
    }
}
'

Delete index

curl -X DELETE "localhost:9200/school?pretty"

Get Index

Get the mappings and setting with the following

curl -X GET "localhost:9200/school?pretty"

curl -X GET "localhost:9200/school/_mapping?pretty"

curl -X GET "localhost:9200/school/_settings?pretty"

Checks if an index exists

curl -I "localhost:9200/twitter?pretty"

Get the count of Index

curl -I "localhost:9200/twitter/_count?pretty"

Update index settings API

curl -X PUT "localhost:9200/school/_settings?pretty" -H 'Content-Type: application/json' -d'
{
    "index" : {
        "number_of_replicas" : 2
    }
}
'

Get the Statistics of the index

curl -X GET "localhost:9200/school/_stats?pretty"

curl -X GET "localhost:9200/_stats?pretty"

curl -X GET "localhost:9200/index1,index2/_stats?pretty"

Congratulations! We successfully completed the basics in elasticsearch.