DEV Community: karenpanahi

proxy with wget

karenpanahi — Fri, 12 Nov 2021 17:09:20 +0000

https://askubuntu.com/questions/346649/how-do-i-force-wget-to-use-a-proxy-server-without-modifying-system-files

https://www.thegeekdiary.com/how-to-use-wget-to-download-file-via-proxy/

follow:

Without modifying your system files at /etc/wgetrc you could create a file at your home directory ~/.wgetrc. It will enable you to modify wget settings locally at user level. Now write the following at ~/.wgetrc to use wget behind a proxy server:

use_proxy = on
http_proxy =  http://username:password@proxy.server.address:port/
https_proxy =  http://username:password@proxy.server.address:port/
ftp_proxy =  http://username:password@proxy.server.address:port/

If you do not have proxy username and password just write the proxy-address and port everywhere like,

http_proxy =  http://proxy.server.address:port/

example :

export http_proxy=http://110.232.67.44:55443/
export https_proxy=http://110.232.67.44:55443/
export ftp_proxy=http://110.232.67.44:55443/

for checking the environement variables you can do :

env | grep proxy

This should work. I am assuming you already have proxy settings for all other applications.

mysql docker config

karenpanahi — Sat, 30 Oct 2021 13:22:58 +0000

https://phoenixnap.com/kb/mysql-docker-container
https://towardsdatascience.com/connect-to-mysql-running-in-docker-container-from-a-local-machine-6d996c574e55

sudo apt-get install mysql-client

docker run --name=mysqlDocker --env="MYSQL_ROOT_PASSWORD=yourRootPassword" -d mysql/mysql-server

now you have to change mysql root user host to accept all ips:

docker exec -it <containerName> bash
mysql -u root -p 
# enter the password you entered in the container config

# in mysql bash enter:
update mysql.user set host='%' where user='root';

Configure Mysql Container

you have to use a .cnf file or you will have this error :

Host 'xxx.xx.xxx.xxx' is not allowed to connect to this MySQL server

create an alternative config file on the host machine and mount them inside the container.

1.First, create a new directory on the host machine:

sudo mkdir -p /root/docker/[container_name]/conf.d

2.Create a custom MySQL config file inside that directory:

sudo nano /root/docker/[container_name]/conf.d/myCustom.cnf

3.Once in the file, you can add lines with the desired configuration.

For example, if you want to increase the maximum number of connections to 250 (instead of the default 151), add the following lines to the configuration file:

[mysqld]
bind-address  = 0.0.0.0

4.change the location of the data directory :
4.1. Find an appropriate volume on the host and create a
data directory on it:

sudo mkdir -p /storage/docker/mysqlData

4.2. Save and exit the file.

For the changes to take place, you need to remove and rerun the MySQL container.

docker run \
--detach \
--name=[container_name] \
--restart unless-stopped
--env="MYSQL_ROOT_PASSWORD=my_password" \
--publish 66030:3306 \
--volume=/root/docker/[container_name]/conf.d:/etc/mysql/conf.d \
--volume=/storage/docker/mysqlData:/var/lib/mysql \
mysql

now you can use either dockerized phpmyadmin or workbench on your host machines.if you want to use phpmyadmin do the following steps otherwise you can jump to step 7.

dockerized phpmyadmin

docker volume create phpmyadminVolume
phpmyadmin-volume

docker run --name mk-phpmyadmin -v phpmyadminVolume:/etc/phpmyadmin/config.user.inc.php --link mk-mysql:db -p 82:80 -d phpmyadmin/phpmyadmin

Access MySQL via phpMyAdmin

Open your browser and visit http://localhost:82 to access phpMyAdmin UI.

Mysql workbench

leave the host option to localhost, in the port section enter the port you exposed for mysql container(in our case 6603) .user and password are the ones you specified in the above section (in our case root ).

bonne chance !

Tree command linux

karenpanahi — Tue, 26 Oct 2021 10:22:57 +0000

really simple tut to get you going with tree command

tag meanings :

tree -a :all 
tree -f :list wit fullpath prefix
tree -d :directories 
tree -df :dirs with fullpath prefix !
# this is the depth the tree goes in(2 means 2 directories) 
tree -L <desired lenght> >>>tree -L  2 
-u: the username
-g: the groupname 
-p:permissions 
-s :size
-h: for showing the size with K ,M ,T
tree -L 3 -pug

wild card

wildcard command would be -P (pattern) :

tree -f -P cata* 
$prune the empty dirs:
tree -f --prune 
$print with permissions 
tree -f -p

Maybe one day ...

karenpanahi — Sun, 24 Oct 2021 22:53:17 +0000

table of content

Nextcloud setup with docker

Nextcloud setup

helpful url:
https://www.cloudsavvyit.com/12476/how-to-self-host-a-collaborative-cloud-with-nextcloud-and-docker/

docker run -d -p 8082:80 nextcloud

link a database container :

--link my-sql:mysql

and then use the mysql as the database host on the setup.

docker basics ubuntu

karenpanahi — Sat, 23 Oct 2021 10:58:07 +0000

table of content :

Mysql configuration
Docker Network
Using ipv6
docker compose
docker data persistence and file management

Mysql docker conf

https://phoenixnap.com/kb/mysql-docker-container

mysql

https://docker-curriculum.com/#webapps-with-docker
https://www.tutorialspoint.com/docker/

tut:

benefits :
no environmental configuration needed on the server Exept
Docker Runtime
if we use the same app but different versions they will use the same port ,how does that work??
-the containers use the same port but the port that the host binds to them are different for example redis:latest & redis:4.6 both use port 6379 but the ports that host assigned to them are different (i.e 3000 & 3001)
but you have to use the host port for each container :

someApp://localhost:3000
someApp://localhost:3001

but if you dont specify the ports while runnuig the container ,it would be unreachable for other apps so :
sudo docker run -p:
sudo docker run -p6000:6379

postgres:version
if not specified
for example: postgres:9.6
sudo docker -d :detached mode

some basic commands

sudo docker logs containerId
if you want to give container a name:
sudo docker docker run -p6000:6379 --name
docker run --name heshmat redis

$ lists the processes running within a container.
docker top container options

$ Display a live stream of container(s) resource usage statistics.
docker stats options container

$ applying limitations:
docker container run -d --name testContainer
--cpu-shares 512 --memory 128M -p 8080:80 nginx

$ if you have an existing container you can update it by:
docker container update --cpu-shares 512 ...

for debugging:
sudo docker -it containerId /bin/bash

start &run :
run creates new
start restarts already made container
sudo docker image
sudo docker run
sudo docker pull
sudo docker run -it packageName /bin/bash
in the above command we have:
-it which runs the (centOs) in interactive mode
/bin/bash which runs bash shell once packageName(centOs) is running

TAG − This is used to logically tag images.
Image ID − This is used to uniquely identify the image.

sudo docker rmi ImageId #removing an image
sudo docker images -q #just ids !
sudo docker inspect ImageName # returns bunch of info about the ImageName

sudo docker run -it busybox sh
interactive mode with

docker container prune #this deletes all stopped containers .

the ps cmd is abbr of process status and is available inside a container
-f ,--filter :filter output based on the condition provided
(here is -f status=exited)
--quiet ,-q :just the id's
--all ,-a :running and stopped containers
sudo docker rm $(sudo docker ps -a -q -f status=exited)

the dollar sign pass the output of the other cmd to previous

sudo docker run --rm prakhar1989/static-site
$# the --rm deletes the container after it has been exited

sudo docker run -d -P --name static-site prakhar1989/static-site
$#-P (capital p) publish all exposed ports to random ports.
$#-D :run the docker on debug mode
$#-

docker port static-site
output >>>
80/tcp -> 0.0.0.0:32769
443/tcp -> 0.0.0.0:32768

containers

sudo docker top containerId
$ this shows the running process
sudo docker stop containerId
docker rm containerId
sudo docker stats containerId
sudo docker kill containerId
sudo docker pause conatainerID
sudo docker unpause containerId

making Dockerfile :

$# specifying the base image
FROM python:3

$# set a directory for the app
WORKDIR /usr/src/app

$# copy all the files to the container
COPY . .

$# install dependencies
RUN pip install --no-cache-dir -r requirements.txt

the next thing wew need to specify is the port number that needs to be exposed.since flaskis runnig on port 5000 thats what we will indicate .

EXPOSE 5000

then we write the command for running the application :
CMD ["python","./app.py"]

building the image:

docker build -t / .
(the period at last is important )

Docker network

user-defined network:

docker network create <networkName>
docker network rm <networkName>

disconnect the containers to the network you want
to remove

connect a container to a user-defined network

 docker create --name my-nginx \
  --network my-net \
  --publish 8080:80 \
  nginx:latest

to connect a running container to userdef network:

docker network connect <networkName> <containerName>
docker network connect my-net my-container 
docker network disconnect myNet myContainer

the above command connect/disconnect an already existing container to an already existing network

Using ipv6

you have to enable the option on the docker daemon and reload its configuration before assigning any kind of network to it.

enable forwarding from docCont to the outside world

By default, traffic from containers connected to the default bridge network is not forwarded to the outside world. To enable forwarding, you need to change two settings. These are not Docker commands and they affect the Docker host’s kernel.

1.configure linux kernel to allow IP forwarding

sysctl net.ipv4.conf.all.forwarding=1

change policy for iptables FORWARD form DROP to ACCEPT

sudo iptables -P FORWARD ACCEPT

ATTENTION: these settings do not persist across a reboot so ,you may need to add them to a start-up script .

overlay networks

The overlay network driver creates a distributed network among multiple Docker daemon hosts

Start the registry automatically
If you want to use the registry as part of your permanent infrastructure, you should set it to restart automatically when Docker restarts or if it exits. This example uses the --restart always flag to set a restart policy for the registry.
$

Tag the image as localhost:5000/my-ubuntu. This creates an additional tag for the existing image. When the first part of the tag is a hostname and port, Docker interprets this as the location of a registry, when pushing.

DATA manging in docker

1-use Data Volumes
docker volume --help
2-Mount host directory (bind mounts)

-the bind mount or volume writes the data to the host filesystem as tmpfs mount write the data into host memory .
-Volumes are stored in a part of the host filesystem which is managed by Docker (/var/lib/docker/volumes/ on Linux). Non-Docker processes should not modify this part of the filesystem. Volumes are the best way to persist data in Docker.
-Bind mounts may be stored anywhere on the host system. They may even be important system files or directories. Non-Docker processes on the Docker host or a Docker container can modify them at any time.

Use data Volume :

create a data volume:
docker volume create my-vol
view all data volumes:
docker volume ls
inspect a data volume :
docker volume inspect my-vol

--mount
docker run -d \
--name=nginxtest \
--mount source=nginx-
vol,destination=/usr/share/nginx/html \
nginx:latest

volumes arent controled by docker meaning that if you delete a container the volume is still there. for deletion :
docker volume rm my-vol

unknown or tangling volumes are a mess .get rid of them by:
docker volume prune

$ docker run -d -P \
--name web \
# -v /src/webapp:/opt/webapp \
  --mount type=bind, source=/src/webapp,target=/opt/webapp
training/webapp \




    python app.py

The above command to load the host /src/webapp directory into a container /opt/webapp directory. --mount parameter throws an error if the local directory does not exist.
--mount also can mount a single data volume.
You can also set readonly volume as --mount source=nginx-vol,destination=/usr/share/nginx/html,readonly.

Docker network configuration

https://jstobigdata.com/docker-network-configuration/

-p is used to specify hostPort:containerPor .
-P Is used to map any port which is between 49000 to 49900 into the container open network port.

for seeing the port :
sudo docker port containerName

use this format for port specification:
docker run -d -p hostPort:theContainerPort
you can use this to run the command:
curl localhost:
curl localhost: 5000
random:
docker run -d -p 80.674.839.82:5000:5000
you can specify it to use udp ,but the default is tcp:
docker run -dp localhost:port:port/udp container ....

Docker network configuration – docker allows network services for externally accessing containers and container interconnections. There are also open source network management tools for docker. Make sure you have a good understanding of the ecosystem before reading this article, check out the introduction.

Will be using docker network to establish a connection, I do not recommend using --link (if you are already using it, please stop).

Copy
1
docker network create -d bridge my-bridge-nwk
-d – Parameter is used to specify the docker network types, as explained above, bridge, overlay, macvlan and etc.

Container connection
Run 2 containers and connect them using the new bridge my-bridge-nwk.

Copy
docker run -it --rm --name busybox1 --network my-bridge-nwk busybox sh
1
docker run -it --rm --name busybox1 --network my-bridge-nwk busybox sh
Open another terminal and run the below code,

Copy
docker run -it --rm --name busybox2 --network my-bridge-nwk busybox sh
docker run -it --rm --name busybox2 --network my-bridge-nwk busybox sh
1
docker run -it --rm --name busybox2 --network my-bridge-nwk busybox sh
If both the above code was sucessful, try pinging one container from another, like ping busybox2 from busybox1.

/ # ping busybox2

NOTE: For multi containers that need to connect to each other,Docker-Compose is recommanded.

Edit network configuration file

Docker 1.2.0 onwards, it is possible to edit the container’s /etc/hosts, /etc/hostnameand and /etc/resolv.conffiles.

Docker Compose

helpful url:
https://docs.linuxserver.io/general/docker-compose

sample docker-compose.yml file :

version: "2.1"
services:
  heimdall:
    image: linuxserver/heimdall
    container_name: heimdall
    volumes:
      - /home/user/appdata/heimdall:/config
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    ports:
      - 80:80
      - 443:443
    restart: unless-stopped
  nginx:
    image: linuxserver/nginx
    container_name: nginx
    environment:
      - PUID=1000
      - PGID=1000
      - TZ=Europe/London
    volumes:
      - /home/user/appdata/nginx:/config
    ports:
      - 81:80
      - 444:443
    restart: unless-stopped
  mariadb:
    image: linuxserver/mariadb
    container_name: mariadb
    environment:
      - PUID=1000
      - PGID=1000
      - MYSQL_ROOT_PASSWORD=ROOT_ACCESS_PASSWORD
      - TZ=Europe/London
    volumes:
      - /home/user/appdata/mariadb:/config
    ports:
      - 3306:3306
    restart: unless-stopped

you can simply run :

docker-compose up -d

from within the same folder and the heimdall image will be automatically pulled, and a container will be created and started.

docker-compose example

helpful url:
https://docs.docker.com/compose/gettingstarted/

1-create a directory
2-create an app.py in the dir with this context :


import time

import redis
from flask import Flask

app = Flask(__name__)
cache = redis.Redis(host='redis', port=6379)

def get_hit_count():
    retries = 5
    while True:
        try:
            return cache.incr('hits')
        except redis.exceptions.ConnectionError as exc:
            if retries == 0:
                raise exc
            retries -= 1
            time.sleep(0.5)

@app.route('/')
def hello():
    count = get_hit_count()
    return 'Hello World! I have been seen {} times.\n'.format(count)

3-create a file requirements.txt in your project directory and paste this :

flask
redis

Docker data persistence and file management

docker volumes

docker volumes are handled by docker and therefore independent of both directory structure and OS of the host machine.when we use volume , a new directory is created within docker's storage dir on the host machine. and docker manages that directory's contents.

Use cases

if you want your data to be fully managed by docker and accessed only through docker containers, volumes are the right choice.

if you need full control of the storage and plan on allowing other processes besides Docker to access or modify the storage layer ,bind mounts could be the right choice but you have to consider the security risks explained here.

Getting started using volumes

the docker daemon store data within the docker directory
/var/lib/docker/volumes/...
Let’s say you want to create a PostgreSQL container, and you are interested in persisting the data. Start with a folder called postgres in :

$HOME/docker/volumes/postgres

docker run --rm --name postgres-db -e POSTGRES_PASSWORD=password --mount type=volume,source=$HOME/docker/volumes/postgres,target=/var/lib/postgresql/data -p 2000:5432 -d postgres

Alternately, here is the same command using the shorthand flag -v:

docker run --rm --name postgres-db -e POSTGRES_PASSWORD=password --v $HOME/docker/volumes/postgres:/var/lib/postgresql/data -p 2000:5432 -d postgres

Get dokuwiki running with docker ubuntu20.04

karenpanahi — Thu, 21 Oct 2021 09:16:08 +0000

create a docker volume :

for data persistency container needs one volume on the docker host ,which will be mapped to /config folder within the container. create the volume by :

    docker volume create dokuwiki

creating the container

docker create \
--name=dokuwiki \
-e PUID=1000 \
-e PGID=1000 \
-e TZ=America/New_York \
-e APP_URL=[YOURURL] `#optional` \
-p 8081:80 \
-v dokuwiki:/config \
--restart unless-stopped \
linuxserver/dokuwiki

variable explenation:

-e PUID : set process userid to 1000 to help avoid permission issues

-e PGID : Sets Process Group ID to 1000 to help avoid permission issues.

-e TZ : time zone (with this format: America/Newyork)

run the container :

    docker start <containerId>

if you followed the above configuration then you're done go to http://localhost:8081/install.php for admin config

but if you want to update the already running container :

    docker update --restart unless-stopped container_id
    # verify the changes you made : 
    docker inspect container_id

and search for RestartPolicy ,this is what you will be searching for :
"RestartPolicy": {
"Name": "unless-stopped",

you are done !
go to this URL :http://SERVER-IP:8081/install.php for admin configuration .

this might also be helpful:

https://codeopolis.com/posts/how-to-install-dokuwiki-on-docker/