DEV Community: technonotes-hacker

DAY 3 - SAGA Design Pattern

technonotes-hacker — Thu, 24 Sep 2026 03:49:04 +0000

To manage distributed transactions across multiple microservices.
It divides a large transaction into a series of smaller local transactions.
Executed independently.
Each service performs its own transaction via events or commands.
If one transaction fails --> compensating actions are executed to undo previously completed operations.

Distributed Transaction

Multiple db or system working together to complete a single task.
If any step fails --> the entire transaction may need to be rolled back.

Why SAGA

To manage distributed transaction in microservices without using complex protocols like 2PC.

All these are achieved because they are executed independently.
2PC is notorious for blocking resource locks over a network, creating performance bottlenecks and single points of failure in high-throughput banking systems, whereas Saga decouples local transactions and relies on compensation.

Working of SAGA

Distributed transactions by dividing them into a sequence of smaller transactions that execute independently across multiple services.

Example

Flow

Approaches

Choreography-Based Approach (Event-Driven)

No central coordinator
each service knows what to do ( independent )
Events based via message queues or event streams.
If a service fails --> it publishes a failure event accordyingly perform compensating actions.

Orchestration-Based Approach (Centralized)

SAGA Execution Coordinator (orchestrator) controls the flow.

The orchestrator tells each service when to start, what to do, and when to proceed to the next step.
In choreography, services listen to and emit events without a central coordinator. As the system scales, understanding the exact workflow path becomes complex, and accidental event loops can occur.

Advantages

flexibility & scalability --> to execute transactions independently without global locking.
NO single points of failure since transactions are distributed across multiple services.
fault tolerance --> ompensating actions to handle failures gracefully.
Sagas embrace eventual consistency. The system transitions through intermediate states, and good banking application UX transparently reflects this operational reality to the end user.

Dis-Advantages

Increase system complexity due to distributed transaction handling and compensating operations.
Consider how communication happens when there is no "conductor" or central manager directing the traffic between microservices

Notes :

2PC --> two phase commit

Questions

Kubernetes - Day - 06 - Network Inside Docker

technonotes-hacker — Wed, 23 Sep 2026 18:19:50 +0000

Different types of Network

Why we need to speak about network ?

How a container speak to anothe container ?
How a container connect to internet ?

To answer all these questions we need to study different types of Network.

Bridge Network ( default )

docker0 ( which is the default bridge )
172.17.0.0/16 172.17 --> Blocked to Network 0.0/16 --> Available to the Host
Uses IP tables
docker 1 runs nginx & docker 2 runs postgres.
Default bridge --> you can hit docker 1 to the other docker by IP not by its name.
Container gets random IP's.
They can't ping each other by name ( no DNS ), only by IP address. Because of this custom network exists.

docker run -d --name web_nginx nginx docker run -d --name db_test postgres docker inspect web_nginx

Bridge Network ( Custom )

docker run -d --name web_nginx --network mynetwork nginx docker run -d --name web_nginx --network mynetwork postgres docker exec -it web_nginx bash docker inspect db_test

Now we can see DNS in the inspect command.
Inside my give network , i can speak via Domain Name. No need to know the IP. (this is advantage of the custom Bridge Network).
Every user defined network gets automatic DNS.
Containers reach each other by NAME, not by IP which is the recommended setup.

Host Network

The container shares the host IP & port space entirely.
Fasted option but no isolation.
Port clashes are possible.

docker run -d --network **host** nginx ( host --> its runs as if like host )

Instead of creating a separate environment , just run in my host itself.
It will run as a process or a software.
There won't be any IP allocated.

None

You run a container , it can't communicate to the outer world.
It can used to run Any task.
No request can be sent inside.
It will be manily used for Job / cron tab.
Its an isolated machine.
Volume mount can be done and necessary packages can be run as container.

docker run -it --network none alphine bash/sh
docker run -it --network none python:3.10.21-alphine3.23

Overlay

Its like tunnle.

MacVLan

A conatiner gets a REAL IP & MAC address on the physical network.
Other networks won't have MAC address , only MacVLan will have the MAC address.
It will be like " ஒரு laptop பக்கத்துல இன்னொரு laptop இருக்கிற மாதிரி ".

ip addr ( check the IP of the machine ) Eg., 192.168.1.5/24
docker network create -d macvlan --subnet=192.168.1.0/24 --gateway=192.168.1.1 -0 parent=wlp2s0 macvlan
docker run -d --network macvlan_net
docker exec -it <number> bash
docker inspect <number> ( IP will be 192.168.1.2 )

Eg., It will be used for legcy system to communicate eacf other.

Commands

docker rm -f $(docker ps -aq)
docker run -d --name web_nginx nginx
docker run -d --name db_test postgres
docker inspect web_nginx ( it will show the default bridge network )
docker logs -f db_test
docker network ls ( list the set of networks available )
docker network create mynetwork ( its takes the default Bridge network )
docker run -d --name web_nginx --network mynetwork nginx
docker run -d --name web_nginx --network mynetwork postgres

Notes

What is Redis ? Redis is a free, open-source, in-memory data structure store used as a database, cache, message broker, and streaming engine.

DAY - 2 - Encryption Technologies

technonotes-hacker — Wed, 23 Sep 2026 04:18:08 +0000

Cryptography Fundamentals

Symmetric Encryption

Its a Private Key.
Single shared secret key for both encryption and decryption.
Mainly used for bulk data processing.
Algorithm used : AES (Advanced Encryption Standard) and 3DES ( Triple Data Encryption Standard ).
3DES is migrated to AES under current RBI cybersecurity guidelines.

Asymmetric Encryption

Its a Public Key.
Uses a mathematically linked key pair (Public Key & Private Key)
Data encrypted with the Public Key can only be decrypted by the corresponding Private Key.
Algorithms include RSA and ECC (Elliptic Curve Cryptography).

Public Key Infrastructure (PKI)

PKI binds public keys to real-world identities using Digital Certificates issued by a trusted Certificate Authority (CA).

Key Management

Private Key must never be shared or transmitted.
It should ideally be stored inside a tamper-resistant Hardware Security Module (HSM).

Hashing

one-way, irreversible function that takes an input and generates a fixed-size string.
Used for data integrity.
Algorithms: SHA-256, SHA-3

Digital Signatures

Provides Authentication, Non-repudiation, and Integrit.
The sender hashes the message and encrypts the hash using their Private Key.
The recipient decrypts it using the sender’s Public Key to verify identity.

Data Protection in Banking Environments

Data at Rest

Stored data in databases, SAN, core banking systems.
Protected using AES-256 Transparent Data Encryption (TDE).

Data in Transit

Data moving across networks.
Secured via SSL/TLS (Transport Layer Security).
TLS 1.2 or TLS 1.3.

Tokenization

Replaces sensitive data (like a 16-digit Debit Card number) with a non-sensitive equivalent called a Token.
The original data is stored in a secure centralized token vault. Essential for PCI-DSS compliance.

Data Masking

Modifies the data structure so it remains unreadable to unauthorized users but retains its format (e.g., masking a bank account number as XXXXXX1234).
It can be Static (altering data in a test database environment) or Dynamic (altering it on-the-fly based on user role privilege).

Questions Set 1

Which international standard governs PIN block formatting (such as ISO Format 0, 1, 3, or 4) during ATM and POS transaction messaging? ISO 9564 defines the PIN management and security, including the exact structures for formatting PIN blocks.

Questions Set 2

Notes

"Inherently provide" means that a feature is naturally, automatically, and permanently built into a system.
What is the one security feature you get AUTOMATICALLY from a Digital Signature, which you CANNOT get automatically from basic symmetric encryption (like AES or 3DES)?

Non-repudiation means that a person or entity cannot deny the authenticity of their signature on a document or the sending of a message that they originated. You cannot say 'I didn't do it' when you actually did.
Digital Signatures use asymmetric encryption (Unique Private Keys) and are the only tool that inherently provides non-repudiation.
Vulnerability Assessment and Penetration Testing - VAPT
Asymmetric encryption is computationally expensive for large files. How does reducing the data size help?
Online Certificate Status Protocol (OCSP)
Certificate Revocation Lists (CRLs)
Which apex body provides the digital certificate infrastructure and secure network backbone for RTGS transactions in India? IDRBT stands for Institute for Development and Research in Banking Technology.IDRBT publishes the "Cyber Security Checklist for Banks" and frameworks for cloud computing, mobile banking security, and biometric authentication.
1 Round Trip Time (1-RTT)
PCI-DSS ??

Kubernetes - Day - 05 - IP Address & Subnetting

technonotes-hacker — Mon, 21 Sep 2026 03:37:01 +0000

IP

IP is just an address.
Every Server is an IP.

Public IP Range
Private IP Range

Breaking the Public network to different subnet.
/24 ( This refer to subnet )
Private IP is just a namespace.
All these are IPv4.

I can restrict/seggregate the subnet.
/32 is the last.

Notes

Subnet means Private.

Kubernetes - Day - 04 - Docker Commands

technonotes-hacker — Mon, 21 Sep 2026 02:11:36 +0000

IMAGES

docker images

alpine images & bookworm images ( these are like families ) --> these are small images to be used for 10 mb application rather than going for bigger version of OS.

docker pull ubuntu:noble-20260810

docker ps [ process status ]
ps aux [ process which are running currenlty ]
docker ps [ currently running containers ]
docker ps -a [ provides the list of images which are Exited ]
docker rm $(docker ps -aq)

docker run ubuntu 
docker ps -a [ Now , this will show as Exited ]

Why ? "docker run ubuntu" --> This command didn't tell anythig to do. So it will be EXITED.

docker run ubuntu ls
docker ps -a [ Now , this will show as Exited with ls ]

If you want to create a IMAGE , then there should be always a entry-point.

docker run -d ubuntu bash         [ D for daemon ]
docker run -it ubuntu bash        [ bash is a shell command ]
docker exec -it  <container_id>   [ exec means execute ]
docker run -it ubuntu sleep 10

11 characters will be displayed in container ID.

docker pull redis
docker run redis
In another terminal --> docker ps -a
In another terminal --> docker exec -it <container_id> bash

docker run -d --name webapp-nginx nginx 
docker ps [ it will be showing UP ]
docker exec -it <container_id> bash
docker stop <name or container_id>

nginx will run in 80 port by default.

docker run -d --name webapp -p 8090:80 nginx
docker exec -it <container_id> bash

Inside the container --> curl http://localhost:80 --> will get the response.
Outside the container --> curl http://localhost:8090

Whatever its inside the container , it gets exposed to the host outside via pot mapped.

Logs

docker logs -f <container_id or name>  [ Continuous logs ]
docker stop <container_id or name>

Bind

docker run -d --name webapp -p 8090:80 -v "./html":"/usr/share/nginx/html" nginx

docker run --rm -d --name webapp -p 8090:80 -v "./html":"/usr/share/nginx/html" nginx [ If the container is stopped , don't wait for my signal , yiu automatically delete it , so only we are using **--rm** ]

docker inspect <container_id or name>

There will be one private ip assigned.
Run the curl with the private IP , it will load the page.
In host , curl http://:80/ & curl http://:8090/

Docker Basic Command

Docker Images Command

Docker Containers Command

Docker Networks Command

Docker Volumes Command

Docker Compose Command

Dockerfile Directives Command

Docker Cleanup Command

Docker Miscellaneous Command

Notes

Redis , Postgress --> will have base OS , as these are application.
Entry-Point, Command & Expose. --> Important Terminology

Kubernetes - Day - 03 - File Permissions, Bind Mount

technonotes-hacker — Mon, 21 Sep 2026 00:51:32 +0000

File Permission

- chmod 600 <file_name>
- chmod -R 6-- <folder_name>
- R --> Recurrsive

What is UMASK ?

This is for default permission.
just type "umask".
Reply 0002 ( what it means ? )
Don't take the first 0. It will be for directory.
It will give 666 (rw- rw- rw-) --> 666 - 002 --> 664.
The result is ( rw- rw- r-- )

Bind Mount

mount --bind <src_folder> <dest_folder>

A folder is like HOST.
B folder is like Container.
This is literally called as Volume.
Bind Mount is nothing buta VOLUME.

Commands

docker run -v "./data":"/home/data" ubuntu bash

Architecture

Image is RO.
No in-place updation for image , only new image can be created.
Each image will have different layers.

Notes :

Container is nothing but spinning of an image using Docker.

Kubernetes - Day - 02 - PID/Signals/Mount

technonotes-hacker — Sun, 20 Sep 2026 14:11:18 +0000

Docker

Pull the details from docker register.

docker pull nginx

its hub repository.
Want to run the image which is pulled.

docker run -d --name web-demo-test nginx [ You will get the ID ]

--> d for detatched mode
--> Conatiner is nothing but THE RUNNING INSTANCE OF THE IMAGE.
--> From one image , we can run many instance.

docker ps [ all the alive running containers ]
docker ps -a [ all the running containers , NOT RUNNING but exited ]

--> -ps [ process status ]

What is process ?

Running instance of a program.

what is program ?

Its in storage/memory, once it runs goes to RAM.
In RAM, some slot will be assigned to that process.

--> First running process will be allocate PID as 1.
--> No parents will be there for 1.
--> all other process will be the child of it.

ps aux [ snapshot of all process ]
pstree [ In tree structure ]
pstree -p [ PID will be displayed ]

--> docker is a container.
--> Inside docker , some application is going to RUN , that will also have the PID.

docker -ps a
ps aux | grep nginx
docker exec -it <container id from docker -ps a> bash

Inside the container when I try to install ps ( sudo apt install procps ) then PID is assigned.
Now nginx is running inside the container with different PID , and in the host , when we check with PID its running with different PID. But the container is running inside the HOST only.

Now Lets try to kill,

kill the nginx with PID in the host.
When we check "docker ps -a" its shows like "Exited".
That means, the process which is running inside the container is using the HOST machine kernel.

docker images [ Display list of images ]
docker rmi <image id> [ It won't delete if the image is running ]

What is namespace ?

PID is a namespace.
its like a separation from others.
even a domain name is a namespace.

- docker inspect --format '{{ .state.Pid }}' web-demo-test [ This will give the process id ]
- ls -l /proc/<pid>/ns

SIGTERM

Terminate --> Graceful shutdown --> 15.
Forefull termination --> 09 --> No clean up process --> SIGKILL.
control + c --> 02
Without stopping the application or long running daemon process , ssh / nginx , we can update without restarting --> 01 -->SIGUP.
Pause a process --> 19
SIGCONT --> 18

- docker logs -f <container_id> [ To get the logs of the container ]

Create a small python program in container like below , import time while True: print("Hi") time.sleep(1)

FILE PERMISSION / BINDING

Mount ? What is mount ?

In the same concept , we have "LINK" in windows eg., shotcut.
Symlink.
Even RBAC we can set.

Notes :

head --> It will show the
Image is also a OS. Image ஓட running instanceதான் Container.
Container won't have size [ COW ] Copy on write. The Container will refer all from images.
So only we can't delete a image when a container is running. ( When the application in the container creates some files , that only will have the size ).
Daemon process --> running in the back-ground.
systemd --> what is "d" here ? No one knows.
Containerகுள்ள ஓடுற Process ID வந்து நம்ம 1ல தான் start ஆகுது , but அதே process , host machineல , வேற ஒரு process IDல ஓடுது. நம்ப Conatinerர ஏமாத்துறோம் Indirect ha. Internalலா , host machine kernel is used.

Kubernetes - Day - 01 - Docker Intro

technonotes-hacker — Sun, 20 Sep 2026 08:33:13 +0000

Why Docker ?

What Docker resolves ? --> Lets see , before that lets see some in-depth concepts.

CONTAINERS

App
Base OS what you need ( Mostly it will be linux flavour )
Requirements ( packages )
what to run.
Even Python can be a base OS.

With these entire item we can make a package and we can build a image and deploy.

With this image --> spawn into CONTAINER.
Container is purely LINUX based.
Sometimes it runs in windows , how ? WSL - Windows Subsystem Linux Layer.
How this is working , in Linux , we have CGROUP , which allows the windows to run or support.
இந்த Container technologyயை ஒரு service கொடுக்கிறது தான் --> Many tools are there , One such company is DOCKER , PODMAN.

In DOCKER --> Docker Engine is Open-Source. If you are planning to use Docker Desktop then you need to pay.
In Podman --> All are Open-Source.

COMMONDS

- docker pull ubuntu
- docker run -it ubuntu bash
- docker ps -a
- docker rm -f $(docker ps -aq)

Notes :

Bare Metal -- > Its like a simple laptop with OS. ஒரு OS அதுக்கப்புறம் Applicationக்கு தேவைப்படுற files அவ்வளவுதான் இருக்கும்.
Image is nothing but OS or the application itself , from image it will become CONTAINER.
Many Images are called Registery.
With 40mb we can spawn a OS- Ubuntu. See below.

TBD :

What is redis ?
Just learn docker engine installation.
What is CGROUGS ?

LEARN

LLM - Day 3 - Prompt Eng 2

technonotes-hacker — Fri, 18 Sep 2026 02:12:30 +0000

REACT Framework ( Reasoning & Action )

LLM is model or a model file.
இப்ப ஒரு LLM Model வந்து என்கிட்ட இருக்கு , அந்த Model வந்து ஏதோ ஒரு set of dataவால train ஆகியிருக்கும் நான் இப்ப query கொடுத்தா அது எனக்கு ஒரு response தரும்.
இப்ப நான் அது கிட்ட போயிட்டு இன்னைக்கு gold rate எவ்ளோ என்று கேட்டால் அது தருமா ? தராதா ?
LLM --> Algorithm and pre-trained data is a Model. Also LLM is a file.

If LLM can't do anything , it will expect an ACTION or some other tools.
Tools are called as programs.
Programs are called scripts.
LLM + Tools ( Tools nothing but a script ).
Can a LLM call another file ? Yes , you can call the tool is AGENT.
Its called TOOL CALLING --> He is nothing but AGENT.

In framework we have RICE Pattern.

For this pattern we give --> Role , Input , Contraints , Expectations will be provided and ask the query.

In framework we have CRISP Pattern.

For this pattern we give --> Context , Role , Instruction , Style , Purpose.
Eg., Blog writing

TOOL CALLING - AGENT

run_example.py file will import all items from the techniques folder and then running each one functionalities.
need to be installed all these to run. pip install -r file_name , requirement file.

To create a Agent " langchanin " is enough.
LLM + Tool --> to orchestrate these two is AGENT.
.env will have API key of the groq. ( which will connect to the cloud LLM ) why we are connecting like this ? Because we can't run the LLM in the local as we are going to use gpt-oss-120b.
Render the messages is also available.

Zero-Shot

run_prompt
**ChatPromptTemplate **is from langchain package.
All will be passed in LIST.
No example will be given for this Zero-shot

Few-Shot

Example will be given.

Tree of thought

Option based [TBD]

Chain of thought
Role based
Instruction based
Contextual Prompting
Self consistency
React

Notes :

Many frameworks are there --> like CARS.
Groq --> Infra provider for LLM.
llama3 & gpt-oss 120B --> these are two models which are predominantly used.
langchain is more stable to create a agent.
Temperature --> based on facts or closely/choice.
Don't memorize the code , because each framework is differemt. Just get the flow.
How the model are choosen ? Hugging face ( they will give the model ) , ollama.
Image --> Blip Model.
Text --> Qwen Model.

LLM - Day 5 - RAG a Intro

technonotes-hacker — Thu, 17 Sep 2026 23:49:11 +0000

Retrievel Augment Generation

3 different Topics
when we need RAG ? You have Private Data Source Eg., கம்பெனி குள்ள இருக்கிற Confluence dataஓ or documents ( these won't be available to public )

The above funcationality should be available for my private data also. Eg., In my company leave policy.
This is resolved by RAG.

RAG Architecture

3 parts are there.
Pre-processing steps --> RAG , ready பண்றதுக்கு முன்னாடி நான் இதெல்லாம் செய்யணும். இந்த diagramல dash dash இருக்குல்ல அதுதான்
Before pre-processing , let me draw a graph.

Orange x pointகு closeசா இருக்கிற point சொல்லுங்க. So [8,9,5] . How you told these point , how you processed ? Going to take distance of all points then sort. This is what we are going to get in Pre-processing.
Blue x are the chunks. what is Chunks ? A part. A part of what ? its the documents.
Eg., 500 character is a chunck. C1 . Then again C2 ... C5 etc , entire documents.எல்லா documentமெண்ட்டையும் துண்டு துண்டா உடைக்கும்.

Now all chunks C1 C2 etc is given to MODEL which is a EMBEDDING Model.
This EMBEDDING Model will give POINT.
Eg., You are very good food industry.
ஒரு ஒரு chunkகும் ஒரு ஒரு point. Related items occur closer together.
These points will be put in graph. You are creating a cluster like , with similar topics together.
EMBEDDING Model is nothing but LLM also it contains Transformer.
In simple -> நீ ஒரு TEXT குடுக்குற நான் ஒரு POINT கொடுக்கிறேன், அவ்வளவுதான் indirectly a Knowledge source.
So they are in MULTIDIMENTIONAL SPACE.
For a single chunk it will be 786D ( it means 786 different points ). To create this dimention , VECTOR DATABASE.
Vector & Embedding are almost similar. ( Here Vector will have direction in addition ).
Before giving to LLM , it will be chunked.
Vector DB eg., softwares below

Orange X is the query.
With this query , give me the TOP-K elements கொண்டு வா .

One is Distance , It will give shortest distance algorithm like ( euclidean algorithm , manhattan algorithm , etc ).
Another is Cosine similarity --> Lesser the angle , these points are more similar or closer to each other. , also its SEMENTIC. what is Sementic ? Occur close together. Eg., Rice and Biriyani , but indirectly they are close together. In simple , how similar these two points are.

Contexts --> Actual data from Vector DB. (this is nothing but the meta data).
Query + context --> you have right now.
Now with this AUGUMENT , its nothing but "PROMPT".
This Prompt will have "Query + context" then format , etc etc all condition. இந்த processசஸ ரெடி பண்றதுக்கு பேருதான் Augument.
Now we will give this to LLM. Eg., chatgpt , gemini.
For this LLM , its a query.
How the dynamic doc are handled ? Recreate or upend. Depends on the documents.

Notes

"Confluence data" or data confluence means தரவு சங்கமம் (Tharavu Sangamam) — the combining of different data sources, streams, or types into one single place or system.
obsidian graph
https://plotly.com/python/3d-scatter-plots/
Augument means COMBINING.
Query is Orange X and Blue is Chunks. Along with the point , a meta data will be stored . DATA ABOVE THE POINT .

LLM - Day 1 - Intro

technonotes-hacker — Thu, 17 Sep 2026 19:19:41 +0000

Large Language Model

Its a model
what is a model ? its a equation.
Then what is equation ? y = mx + c or y = m1x2 + m2x3 + m3x2
Models are nothing but WEIGHTS.
Take an example Eg: gpt-oss-120b , for this model they would have used data with TRANSFORMER ARCHITECTURE and we are getting some output that is called MODEL. m --> weights ( we can adjust the m and get many desierd outputs )

Model is floating point storage.
சில பேரு சொல்லுவாங்க இது வந்து 120B Paramter model , what it means ? These Parameters are nothing but how many weights I am having inside the model. நான் எந்த அளவுக்கு dataவை கொடுத்து train பண்ணி இருக்கேன்னு அர்த்தம். In other words , CONTEXT அதிகம் .
Can we run this 120B Paramter in machine ? --> Where the OS sits ? C drive or /boot , initially it will be in hard disc , once the machine starts , it will go to RAM Running memory போய் உட்கார்ந்து கொல்லும். So to run the model we need RAM and also the CPU , so these two decides which model to run. Eg., in laptop we can run 7B parameter.
In local like laptop we can run eg., llama cpp, ollama , LM Studio . இது எல்லாம் modelஆ ? No , these are softwares/tools to run the model.
Open model are called like " Open weights model ". Eg., chatgpt model , gemini model. ( Referring to Open source )

Why I Need a model ?

LLM is a next word predictor. Eg., Hi How ......... இதுக்கப்புறம் என்ன வரலாம் like are you , is your life , is the food . இது எல்லாம் ஒரு possibilities ( It can have n no of possibilites ). MORE THE PRESENCE , MORE POSSIBILITY OF OCCURING. நீங்க ஒரு வார்த்தையை குடுக்குறீங்க , அதுக்கு அடுத்த வார்த்தை இது கண்டுபிடித்து தர போது . Then this will go as input , like wise it will generate.
All each possibilities will have SCORE , entire score will like between [ 0 - 100 ] or [ 0 - 1 ] .
If you are not giving any controlling parameter , then it will chose the highest SCORE.
If you want to control the LLM , at that place . We have controlling parameters. Namely :

Temparature
Top-k
Top-p

Temparature

0-1
controls how random or safe the model choices.
If the TEMP is low like ( 0-0.3 ) then it will ACT BASED ON THE FACTS. Likely having max score / most occuring one.
If the TEMP is high like ( 0.7 - 1+ ) , model is willing to choice less likely words. Eg., in storing telling or in creating writing. It will choose less likely words from dataset which is trained on.

Top-k

Along with temperature its used.
Model only considers , Top k most items.
K --> Elements

Top-P

Also its called Sampling methods.
It will add the score and then it will work.
Along with temperature its used.

இப்ப Top - pகு 90 % value சொல்லிட்டாங்க . இப்ப அது எப்படி work ஆகும் நா. Each prediction will have a score eg., first prediction 0.70 , அடுத்த prediction 0.15 , அடுத்த prediction 0.30 . It will add the prediction score which is equal to 90 % ( more or less ) இத வச்சு தான் அந்த output கொடுக்கும்.

Tokens

words ?
which are " Not complete ".
Entire text into pieces.
Tokenization ? Process of chopping your text into tokens.
Each token will be assigned numbers.
Take it like a translator.

Context Window

Model ஓட short memory.
Short term memory.
Max of tokens a model can see in a given time.
ஒரு timeல ஒரு model லால , எவ்வளவு விஷயத்தை பார்க்க முடியும்.

TBD :

What is dataset ?
Chunkings ?

Notes :

If I am going to use a model , then I need to unzip it , for this i need a running memory so we need RAM. Now , i need process the data , then CPU & GPU ( training a& usage ) are required.
Weigts will be like FOLATING POINTS or EMBEDDINGS.
https://groq.com/ --> Its a tool --> They give free models for usage.

Fine Tuning is not updating the model or touching the weights. We use like rapper around the model thats called Fine Tunning.
Markov Chain --> this is used in next word prediction in gmail , word etc .
Harry Porter and context window .

Jenkins - 1

technonotes-hacker — Sun, 01 Dec 2024 22:02:31 +0000

Open source integration tool.
It can be made as centralised server by integrating multiple source like source code management , build tools , deployment environment .
Complete INTEGRATION TOOL
Its Java based.
To automate the repeated task.
CI/CD ( continuous integration and continuous delivery )
Even you can do patching.
In simple , its a centralised server.

Advantages

CI/CD
Open source
Community driven
Browser based
Supports all Operating system.
Distributed build --> Master and Slave node , how ? If you are setting up Jenkins , it uses resource of the system where its installed and accordingly the Jobs will take the RAM , CPU and resources. To avoid this "Distributed build" was introduced , so that "Master and Slave node" with that the load will be distributed.
We can increase the scalability of master and slave.

CI/CD Workflow

Commit
Build
Test
Stage
Deploy

Stages will be different in other CI/CD process. Purpose will be the same.
Dedicated branch for each environment it will be a best practice to avoid any confusion here.

Installation & Configuration

sudo apt-get update
java -version

Add the official key of Jenkins and then the repo.

sudo wget -O /usr/share/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2023.key
echo "deb [signed-by=/usr/share/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt-get update
sudo apt-get install jenkins

Now JENKINS had been installed , please check with command also whether its installed or not.

dpkg -l | grep -i jenkins

As we have encountered error , install higher version of Java from 11 to 17 or 21.

sudo systemctl start jenkins
sudo systemctl enable jenkins
sudo systemctl status jenkins

Now open the console with the IP of the system.

If you want to know the username , then go to below location.

cat /var/lib/jenkins/users/users.xml
/var/lib/jenkins/users/admin_17980521444909415742

want to change the password , edit and restart the jenkins.

<hudson.security.HudsonPrivateSecurityRealm_-Details> <passwordHash>#jbcrypt:$2a$10$OTYB2osGPi/rasutjHcYOOhByiCoaEZTEQk52CABOwYdrtxeIPnBu</passwordHash> </hudson.security.HudsonPrivateSecurityRealm_-Details>

Give no password for jenkins user ( as its a internal user )

Check whether able to get output without sudo password.

User Management & security

Manage Jenkins > Security

Trying to use the OS user for security features , lets try.
Add the user to the group shadow like below and restart the Jenkins.

Now its integrated with the OS user (sathishpy1808) , you can even login with the OS users after integrating.

Jenkins own database

User created in Jenkins only can login , OS users can't.

User creation

Change to "Jenkins own database" then only you can view the user creation option like below ,

Go to the ravi user profile and explore.

Even you can terminate all sessions from the console.
You can change the password also and delete option is also available.

Authorisation

what type of authorisation is given to the users , you can see many options in the below screen.

BY default --> "logged-in users can do anything" its equal to admin access --> its not advisable.
Try to use with user based ,

Also ROLE based plugin it will be useful , lets install ( mostly used plugin for role based authorisation )

Manage & roles won't be available at first , it will come after the plugin installation only.

Manage roles and assign roles are very important option.
Lets create a role and then assign the users.
Pattern based users are created only in "Item Roles". Here if any user starts with word "Manage" , it should the managing job roles.

Create some Jobs to test this user specification.

Item roles in "Assign Roles" will be useful for pattern description for users.
You can see all roles which are added.
All users are added in this console.

Job Management

Jobs needs to be created to perform a task.

Build Triggers

"Trigger builds remotely (e.g., from scripts)" --> it will trigger once some other task gets triggered.
"GitHub hook trigger for GITScm polling" --> once the code gets committed then only the job needs to be triggered.

Build Environment

Build Steps

This is the main step , you can add number of steps here.
Raw commands , shell script , etc.
Can be added like stages.

Post-build Actions

Notes

https://www.jenkins.io/doc/book/installing/linux/ --> In this site , you can get the details of the key and the repo details which we have added in the first of the installation part. [TBD]
What is "openjdk-17-jdk-headless package" ?
Default port of Jenkins is 8080.
jenkins user will be always running.

Commands Used

java -version
locate jdk
whereis java

List of errors

Job for jenkins.service failed because the control process exited with error code.

Now you can get the details by typing ,

journalctl -xeu jenkins.service

Error clearly says , ( version not supported )

Dec 01 22:03:57 meet.sathishpy1808.org jenkins[40153]: Running with Java 11 from /usr/lib/jvm/java-11-openjdk-amd64, which is older than the minimum required version (J>
Dec 01 22:03:57 meet.sathishpy1808.org jenkins[40153]: Supported Java versions are: [17, 21]
Dec 01 22:03:57 meet.sathishpy1808.org jenkins[40153]: See https://jenkins.io/redirect/java-support/ for more information.
Dec 01 22:03:57 meet.sathishpy1808.org systemd[1]: jenkins.service: Main process exited, code=exited, status=1/FAILURE

Access Denied

change the ownership to the current user,

sudo chown -R sathishpy1808:sathishpy1808 /var/lib/jenkins/
sudo chown $(whoami) /var/lib/jenkins