DEV Community: yazn zamel

Rethinking ACID: An Inquiry Into Their Fundamental Properties

yazn zamel — Sun, 20 Oct 2024 05:43:18 +0000

ACID

Defining ACID by "THE TRANSACTION" example

Banking systems relies on transaction for money going in and out , they rely on a method called "double entry" , which goes as following : when a user deposit a certain amount of cash a Transaction of addition (++) is made on the liability side indicating the bank owes the customer XX$ amount of money , also another transaction is made on the Assets side which indicate the bank now have more cash. The Assets and Liabilities amounts are stored in a separate account , At the end of the day to ensure accuracy and balance, the bank will subtract the total liabilities from the total assets. In a properly functioning system, this difference must equal zero. This means that every dollar (or other currency) added to the assets has a corresponding entry on the liabilities side, maintaining the balance.

Now, imagine a scenario where multiple failures occur during the processing of online transactions. Unlike physical transactions, which can be easily traced, a lost online transaction can vanish without a trace, leaving no physical record behind. This creates a critical challenge for the bank, as tracking and resolving these issues becomes difficult, potentially leading to significant financial losses and undermining the trust of customers."

Transaction : a series of queries written by a developer to execute a certain logic. The transaction ends once "COMMIT" word is present.

Atomicity

This concept came from the actual atom concept that the atom can not be divided (at least at the past time) , and they applied the same concept on the a database transaction.
Physical atom properties :

All-or-Nothing Nature:

Atom: An atom is the smallest unit of an element that can't be divided without losing its fundamental properties. If an atom is split, it no longer retains the characteristics of the original element.
Transaction: A transaction in a database must be fully completed or not at all. If any part of the transaction fails, the entire transaction is rolled back to ensure nothing partial remains.

Integrity

Atom: Integrity in an atom means it maintains the specific identity of the element (like the number of protons) that defines it. It remains consistent with its element’s characteristics unless it undergoes a fundamental change.
Transaction: Integrity in a transaction ensures the database remains consistent and correct. All changes are applied together to maintain the database’s accuracy. If a transaction fails, it prevents any partial, inconsistent states by rolling back.

Jack : why atomicity , why don't we just keep things simple ?

Author : Since you asked, let's consider a straightforward scenario. Imagine that at the end of the month, your company sends your expected salary to your bank account. Now, suppose the bank, responding to your complaint about things being too complex, decides to simplify by hosting everything on a single database but fails to apply the Atomic property. During the transaction from the company's account to yours, the system crashes and needs to be restarted. Unfortunately, the first part of the transaction—where the money is withdrawn from the company's account—completes successfully, but just as the system is about to deposit the money into your account, it crashes!!

Author : So now what ? do you want to keep it simple ? does it sound simple now to you when you have bills , family , loan ?

yeah man , that's why we care about every detail and not make things complex.

How we as software engineers , Developers or people who care manage such kind of issues ?

When such failure happens , we will have to rollback the partial successfully executed queries (which is the withdrawn from company account) to ensure the atomic property.

what if the transaction was very long that consist of 10s or 100 of queries ? well it's better that we stay safe in such situation. However there is a method in postgresql called 2PC (Two Phase Commit) that can be used in distributed transactions. With 2PC, a transaction is prepared and partially committed in a way that it can be safely completed or rolled back later. This process involves:

Consistency (The sacrificed property)

Jack : now what is this consistency ? we already solved everything with the Atomicity ....

Author : you wish , lemme ask you this : if a transaction is committed with a new change , will another transaction running at the same time or after it in a few see that change ?

Jack : huh ! , just go ahead with your example man.

Alright , following up on the company salary transfer , imagine it's the end of the month and you have promised your son you will get him the new PS5 , once your salary is transferred. Now probably your son is waiting for your salary more than you do. You opened the bank application and still can't find it there , you open it again and there it is !!!. Now you went with him to buy him the ps5 and while you are at the cashier you get the following message "your transaction is declined , due to insufficient balance" , huh ???

Look what happened :

A lot of databases gave up on consistency as a trade-off for performance.

When looking at Consistency , you should consider two points :

consistency of read operation
consistency of referential relationship

consistency of read operation

This type of consistency is usually tied to type of synchronization between databases in Leader-Follower architecture .

The inconsistency arise due to the type of synchronization between follower - leader database , options comes as follow :

Isolation

Isolation :

The capability of a database system to enable multiple transactions to access the same data simultaneously without causing interference with one another.

Isolation is like whether a database want to be social , but it will be vulnerable (not security wise though) and it's somehow something we can configure , it can very social , moderately social , not social.

Very Social (Lower isolation) : Increases the ability of many users to access the same data at the same time

Moderately Social : Somehow in between , wait till your read the third point....

Not Social (Lower Isolation) : Decrease the ability of many users to access the same data (lower concurrency) --> needs more system resources (some stakeholders are fine with it though).

Let's go back to jack , he makes us explain it better.

Jack : i will be chill with this one , let's see what you got now . How can this property save my bank account ?

Author : thanks man , this time i want us to blend to the real world scenarios , in a bank account you have a credit card with certain balance if this credit card goes in negative under 10000$ you either lose some credit card score or you get charged some fee.

How isolation could saves you from unnecessary fees :

Dirty Read : when two transactions (t1 , t2) are accessing the same data , however one of those transaction reads something that is not really written to the database , but promised to be there .

Dirty read : because it read a dirty record

Nonrepeatable read : when two transactions (t1 , t2) are happening the same time , one of the transactions read a value twice (or more) in the same transaction but one or more reads produces different values.

Phantom read : when two transactions (t1 , t2) are happening the same time , one of the transactions insert a row which will be found in the second transaction as it is a value from the void.

You may ask what is the problem with phantom read , the issue resides that while i am doing a transaction the data should be consistent.

Durability

Empower a Commited transaction to survive a system failure. means , i commit - you write to disk.

The question is "isn't this how databases operate ? Once i commit it writes to the disk ?"

The answer to this question could vary depending on the database engine you are using , since some database have been implemented to be highly performing and reduce latency which in fact makes it write to your backend server cache or to the cache of the database server and then writing the data in batches to the disk. This doesn't only make writing to the database fast but in fact makes reading from the database fast as well since the recent written data is still cached.

So the key takeaways from this property is to always check the configurations available for the database and tune it based on your needs and application requirements.

Critical applications where data loss tolerance is near zero requires you to flush committed transactions from the database engine to the disk often.

Keycloak 22.0.5 on ubuntu with Postgresql

yazn zamel — Sat, 11 Nov 2023 10:26:59 +0000

Keycloak is an open-source identity and access management (IAM) service that provides robust authentication, authorization, and security features for applications and services. It allows organizations to easily manage user identities, secure access to resources, and implement single sign-on (SSO) capabilities, enhancing both user experience and security. Keycloak supports various authentication methods and can be integrated seamlessly with a wide range of applications, making it a valuable tool for identity and access control in modern software development.

Keycloak installation on ubuntu 20.04

Since keycloak is built with java , we need to install java in our machine and make sure that java version is compatible with our keycloak version , we are using version 22.0.5

you can switch to the root user to avoid typing sudo each time you run command by "sudo -i" , however i will continue the tutorial using sudo

installing java-17

sudo apt update 
sudo apt install openjdk-17-jdk

Installing keycloak

# any external software should be in the /opt
cd /opt
sudo wget https://github.com/keycloak/keycloak/releases/download/22.0.5/keycloak-22.0.5.tar.gz

sudo tar -xvf keycloak-22.0.5.tar.gz

# create a keycloak user and group
groupadd keycloak
useradd -r -g keycloak -d /opt/keycloak -s /sbin/nologin keycloak

# set the directory ownership 
chown -R keycloak: keycloak 
chmod o+x /opt/keycloak/bin

# now we have given the keycloak user the permission to 
# execute it's binaries

Now we will move the .conf files from the current directory to the /etc

/etc folder is used to save the configuration files

cd /etc
mkdir keycloak

cp /opt/keycloak/conf/keycloak.conf /etc/keycloak/keycloak.conf

# give the keycloak service the ownership to be able to run 
#the kc.sh file
chown keycloak: /opt/keycloak/bin/kc.sh

# create a service in the system 
cd /etc/systemd/system
nano keycloak.service

the keycloak.service file

[Unit]
Description=Keycloak Authorization Server
After=network.target
 
[Service]
User=keycloak
Group=keycloak
ExecStart=/opt/keycloak/bin/kc.sh start
ExecStop=/opt/keycloak/bin/kc.sh stop
Restart=always
RestartSec=3
Environment="JAVA_HOME=/usr/lib/jvm/java-17-openjdk-amd64"
[Install]
WantedBy=multi-user.target

start the keycloak service

systemctl daemon-reload # any change you do on the keycloak.service file run this command after it
systemctl start keycloak.service 
systemctl status keycloak.service

before moving the postgresql part , make sure to install the postgresql client on the keycloak vm

apt install postgresql-client-common

Install Postgresql and connect it with our keycloak

sudo apt update
sudo apt install postgresql postgresql-contrib -y

# now switch to the postgresql user to create the keycloak table
sudo -i -u postgres

psql
CREATE DATABASE keycloak;
CREATE USER keycloak WITH PASSWORD 'admin';
GRANT ALL PRIVILEGES ON DATABASE keycloak TO keycloak;

\q

Now we will configure the postgresql to allow connection from keycloak

#navigate to the postgresql conf file 
cd /etc/postgresql/12/main/
sudo nano pg_hba.conf

# scroll down to the IPv4 local connection and add a new line below the existing one 

host    all             all            <VM-IP>/32          md5

# we will modify the postgresql.conf file to allow the db to listen from other hosts

sudo nano postgresql.conf 
# find the line that have listen_addresses and change it to
listen_addresses = '*'

You can now test the connection locally from the db using this command

psql -h localhost -U keycloak -d keycloak

Configuring the keycloak.service to communicate with the postgresql database

# ssh into your keycloak vm 
cd /etc/systemd/system
nano keycloak.service
# add the following lines
Environment="DB_VENDOR=postgres"
Environment="DB_ADDR=Postgresql_IP"
Environment="DB_DATABASE=keycloak"
Environment="DB_USER=keycloak"
Environment="DB_PASSWORD=yourpassword"

# make sure to change the Environments based on your config

sudo systemctl daemon-realod
sudo systemctl restart keycloak

Note: this way we are running keycloak in production mode

Note: i am assuming you have opened the port for both postgresql 5432 and keycloak 8080 (this might differ)

Hope this helped you set up your environment , for any help feel free and don't hesitate to contact me.