DEV Community: Alfadil mustafa

How to approach your application infrastructure

Alfadil mustafa — Tue, 11 Apr 2023 15:01:47 +0000

Here are some of my thoughts about DESIGN APP'S INFRASTRUCTURE.

MAINLY YOU NEED TO HAVE A PERSPECTIVE, In other words you must know the application you are about to work with. what is it's nature? how it handles the data? does it needs to scale? and so on... .
Setting a perspective will guide you and ease your decision making process.

Some principles for you to follow

The application should be able to start and stop without any corruption on the data.
You should be able to tell that the app is working or not.
You should be able to replicate the app environment.
Re-configuring the application should not lead to re-writing it.
The data should always be external to all the components using it.
Try to reach the ephemeral host concept. meaning that any host should be expendable.

Here are some questions you need to answer

1. How you will handle data? (filesystem, cache and database)

The filesystem

And when I say the filesystem I means all the files that the application uses directly from the host, Using those files in business applications usually leads to data lost or complicate the process of scaling or shuting a server down so I would recommend using network filesystem like (nfs) or storage as a service solutions like S3 or solutions from others vendors.

Cache

Any data you keep for a temporary period of time usually to speed up the transactions or to host some sessions data.
And like the filesystem and for the same reasons it should be external to the servers hosts using solutions like redis and memcached.

Database

What type of database you will need (SQL, NoSQL) will you host it or use a managed service. As a rule of thumb If you don't have sensitive data just use a managed database you will have less headache.

2. How you will scale each component?

Is your environment scalable?
One might get asked this question by that management or an audit but how to answer it?
let's ask another question what does you need to scale and when?
Is it your backend or the database or the whole solution?
there are two ways to decide:

based on resource utilization

Like CPU and RAM or generally hardware resources.

based on custom (metric)

Any custom business event metric like specific pattern of requests that alarm of a huge load requests or an event external to the application.

3. How you handle failure? (in all levels infra - database - data - app)

Here you need to know how to handle disaster and have a clair disaster recovery plan (DR) for all levels of your infrastructure.

4. How you deal with application starting up and shuting down ?

For the app to make sure of the consistency of the data it needs to properly handle the shutting down to release all the connections and streams.

DevOps for Odoo

Alfadil mustafa — Fri, 20 Jan 2023 13:14:35 +0000

DevOps is all about shipping code to a running environment. It got a lot of benefits but the major one is to have consistence environment and stop (it works in my machine excuse).
and as I worked with Odoo for a long period of time I took it as an example to create a simple way to backage developer's code as a container.

disclaimer:

The pipeline created here is not meant to be used in production it's only for simplifying the idea of Devops for Odoo code base.

The goal:

To create a docker image with the latest changes from git.

Tools:

1.Github.
2.Docker.
3.Github actions.

The repo structure is:
Dockerfile
extra-addons
- addon1
- addon2
- addon..
- …….
odoo.conf
requirements.txt
.github
- workflows
- docker-image.yml

The requirements.txt is only for python modules

As in odoo-devops.

The github action will generate an image with the latest and an image with a hash tag from git.
Like dockermanfirst/odoo-14:latest, dockermanfirst/odoo-14:main-228a04c, you can deploy it on any docker host.

What this structure will guarantee:
Consistency of deployment between environments.
Standard way to deploy the odoo version.

Build container images in Kubernetes

Alfadil mustafa — Fri, 28 Jan 2022 23:02:52 +0000

One of the major steps when you are designing how you gonna ship your app is how you will build it and where.

Some people still use conventional ways to ship their apps where the is some Sysadmin dude who does all the deployment process, Some use automation of some kind, and some use a CI/CD pipeline to achieve this.

Here I'm talking about the last type of teams (The cool one 😎) and precisely the step of building Docker images in Kubernetes environment.

I'm assuming that we have:

Dockerfile for the project.
Docker hub account.
Kubernetes cluster.
Kubectl client.

What we will do:

Create a namespace in k8s.
Get authorized to store (push) the resulted image to docker hub (Docker registry).
Deploy a buildkit engine to a Kubernetes cluster.
Build The docker image and push it.

Let's start:
First we should create a namespace
kubectl create namespace images-builder

Then create a Docker hub account.
Go to your terminal and
docker login
then find the docker config file
in linux you will find it in ~/.docker/config.json
it looks like

if you don't find it in that place go search the internet for docker/config.json in the environment you are currently using.

After that we gonna create a secret containing the docker config file we just found using the command
kubectl create secret generic regcred \ --from-file=.dockerconfigjson=/path/to/config.json \ --type=kubernetes.io/dockerconfigjson -n images-builder

Now it's time to deploy the buildkit engine.

save it in a file let's say buildkit.yaml and then
kubectl apply -f buildkit.yaml -n images-builder

Let expose the buildkit engine to use able to use it.

Save it in a file let's say buildkit-service.yaml and then
kubectl apply -f buildkit-service.yaml -n images-builder

We are going to use the engine form our local machine so let's forward the service using
kubectl port-forward service/buildkitd 1234 -n images-builder
And then download the buildkit client from moby/buildkit.
Download the latest version with the version that suits your machine

Extract the files to specific location.

Now we can build the dockerfile using the command.
/path/to/bin/buildctl --addr tcp://127.0.0.1:1234 build --frontend dockerfile.v0 --local context=/path/to/context --local dockerfile=/path/contains/the/Dockerfile

And to build the image and push to the dockerhub
/path/to/bin/buildctl --addr tcp://127.0.0.1:1234 build --frontend dockerfile.v0 --local context=/path/to/context --local dockerfile=/path/contains/the/Dockerfile --output type=image,name=docker.io/username/image-name,push=true

where username is your docker hub username and image-name is the name of the image you would like to use.

My Odoo Development Environment

Alfadil mustafa — Mon, 15 Feb 2021 21:07:06 +0000

First of all it does not has to be the best way but it's the way that works for me and I would like to share it with you.

RUNNING THE SERVICE

Technologies used:

1- Docker more about it.

at the start we need a postgres database server so we create a container using docker.

docker run -d p 5436:5432 -v -e POSTGRES_USER=odoo -e POSTGRES_PASSWORD=odoo -e POSTGRES_DB=postgres --name dev_pg_12 postgres:12

This means we have a pg server exposed at port 5436 so we can access it in localhost:5436

so you want to get on with your life which is running odoo server and develop awesome odoo apps (or you would be wasting your life here my friend 😕)

so you have this project which based on odoo 12 and you have some custom code so you need to create a docker file to tell docker how to create your container.

1 FROM odoo:12.0
2 USER root
3 RUN mkdir /opt/odoo && mkdir /opt/odoo/extra-addons
4 WORKDIR /opt/odoo
5 COPY requirements.txt requirements.txt
6 RUN apt-get update && apt-get install -y python3-pip && pip3 install --no-cache-dir -r requirements.txt
7 RUN chown -R odoo /opt/odoo

What every line means:

1- The first line mean we are going to use odoo:12.0 as the base image so after this step we gonna get all v12 stuff.

2- Switch to the root user (we need the permissions).

3- We need to create directory to store custom modules(you know, what you think you need to add or change in odoo functionality you big shot).

4- Change the working directory (like 'cd' in bash).

5- Copping the requirements file ,I forgot if you have a requirements file for python of course you need to put it at the same directory as the docker file.

6- Just installing the requirements.txt file.

7- Change the owner of the /opt/odoo we created in step 3 cause the service runs with odoo user permissions.

save the file above on with name (Dockerfile) along side requirements.txt file.
if you don't have requirements.txt omit steps 5 and 6.

Now in the same directory open the terminal and run

docker build -t custom_odoo_v12 .

Congratulation you have a working odoo v12 image on your device.

In this step you need to get odoo source, For v12

wget https://nightly.odoo.com/12.0/nightly/tgz/odoo_12.0.latest.zip

now extract it any where in your pc.
and navigate to the odoo directory inside it
the path should be something like

/opt/odoo_12.0.latest/odoo-12.0.post20210215/odoo

next we gonna need an odoo.conf file
let's keep it simple
/etc/odoo.conf

[options]
addons_path = /opt/odoo/extra-addons/
data_dir = /var/lib/odoo

next we gonna run odoo service by creating an odoo container

docker run -p 8082:8069 -v odoo12-dev-web-data:/var/lib/odoo -v path/to/custom/addons:/opt/odoo/extra-addons -v /etc/odoo.conf:/etc/odoo/odoo.conf -v /opt/odoo_12.0.latest/odoo-12.0.post20210215/odoo:/usr/lib/python3/dist-packages/odoo --name odoo12-dev --link dev_pg_12:db -t custom_odoo_v12

what does it means 🤔
1) -p 8082:8069
Mapping odoo port(8069) to port(8082) on the host (your device).
2) -v odoo12-dev-web-data:/var/lib/odoo
Creating a volume to store odoo data (like filestore ...).
3) -v path/to/custom/addons:/opt/odoo/extra-addons
You add the path to your custom addons.
4) -v /etc/odoo.conf:/etc/odoo/odoo.conf
The path to the conf file
5) -v /opt/odoo_12.0.latest/odoo-12.0.post20210215/odoo:/usr/lib/python3/dist-packages/odoo
The path to odoo source (this is optional you need it if you gonna debug the odoo files like set up a debugger or add some print statements 😅).
6) --name odoo12-dev
The name of the container.
7) --link dev_pg_12:db
Linking the PostgreSQL db to odoo as db (this is confusing I know but it's the easiest way to do it 🤷‍♂️ ).
8) -t custom_odoo_v12
The Image we gonna use.

now you can access the service on HTTP://localhost:8082.
to restart the service and see the log just do

docker restart odoo12-dev && docker logs -f odoo12-dev

MANIPULATING THE DATA:

Technologies used:

1- Odooly get it.

You can always access odoo service via the browser like (chrome , firefox ,...), But some highly repeated processes like upgrading modules or reading data from a model or checking users access rights and rules you may need a faster way to do things.
I personally prefer Odooly it's an amazing tool which I depend on daily.
to access odoo via Odooly you just

odooly --server="http://localhost:8082" -d dev-db-1 -u admin -p admin

there are lots of variations but I prefer this one

then you will be inside Odooly terminal just type help and you will find lots of options like
1- env.install("module_name")
2- env['res.partner'].search([],limit=10)
3- env['hr.employee'].sudo(uid).search([]) #uid of a user
Enjoy...

Clean Odoo debugging

Alfadil mustafa — Fri, 15 May 2020 12:21:09 +0000

Did you ever reach a point where your code looks like this:

print("################ here")
'''
bunch of code
'''
print("################ here2", value)
'''
bla bla lala
'''
if(something):
    print("################ here3", another_value)

and you have to force your code to produce an error to make sour it reach that line?

As you might know odoo cli provide debug option see Command-line interface: odoo-bin
and you can debug now

yah...
you want something a little bit friendly

OK as you wish

let's start 🔥

⚠️ we gonna need to have vscode installed in our machine
and then install python plugin you can find it here

First run this command in your terminal

pip3 install debugpy

it's an implementation of the Debug Adapter Protocol for Python check it!

Then go to where you usually start your odoo server. It should be something like this.

python3 odoo-bin --addons-path=paths/to/your/addons

then change it to

python3 -m debugpy --listen 5678 odoo-bin --addons-path=paths/to/your/addons

5678 is just a port number,So chose whatever available port in your machine( this port is only used for debugging and is not related to odoo services )

then open your addons folder in vscode (if you have more than one folder or you want to see your custom code alongside odoo standard addons put them all in a workspace see how!)

you will see some icons in the left sidebar choose the one that looks like this

click on create a configuration (launch.json) file

you should see something like this:

select the third option Remote Attach

you will be asked about hostname and port (just press enter)

vscode will open a new file called launch.json
replace the content of that file with

{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python: Attach",
            "type": "python",
            "request": "attach",
            "connect": {
                "host": "localhost",
                "port": 5678
            }
        }
    ]
}

if you have changed the port number in the running step change it here too.

To start debugging press Start Debugging button as in:

During remote debugging, the debugging toolbar appears as below:

then you can add breakpoints where you want in your python files like this:

and 💥 everything is working now

TL;DR
check here

Run Odoo in docker for local development

Alfadil mustafa — Fri, 08 May 2020 22:01:13 +0000

anyone who is new to Odoo or even experienced developers may have a problem in install odoo in there local devices,So I'm going to show few tips to run odoo using docker.

🔔 first this technique assume that you are running a linux distribution like (ubuntu, debian, redhat, ...).

now let's do it.

we are going to install Docker CE using https://get.docker.com/ if you already have it installed skip this step to check if you have docker install run

docker info

it's going to show details of the current version of docker installed if not run :

curl -fsSL https://get.docker.com -o get-docker.sh
sh get-docker.sh

after that add the your user to the docker group

sudo usermod -aG docker $USER

where $USER is the name of the user

we are going to create a new postgres database server container(don't worry if you don't understand the meaning of the container you can check at docker )

docker run -d -e POSTGRES_USER=odoo -e POSTGRES_PASSWORD=odoo -e POSTGRES_DB=postgres --name postgres_db postgres:10

Now we can start an odoo server

docker run -p 8070:8069 --name odoo --link postgres_db:db -t odoo

this is going to start latest odoo version which is 13 (in the day the post written)

now in your browser go to http://localhost:8070 and 🎉

and to stop it run

docker stop odoo

to start it run

docker start odoo

to run other versions of odoo just change the tag to odoo:11.0, or odoo:12.0
like

docker run -p 8071:8069 --name odoo --link postgres_db:db -t odoo:11.0

🤔 now what about custom addons ❓

💡 we have volumes to add modules to the server.

docker run -v /path/to/addons:/mnt/extra-addons -p 8069:8069 --name odoo
--link postgres_db:db -t odoo