DEV Community: Saad Ali

Understand data science before using LLM into your AI agents

Saad Ali — Mon, 10 Feb 2025 01:37:00 +0000

Exploring data science is crucial, not just for aspiring data scientists. Understanding how data is structured and processed is essential even if you're building and integrating AI agents. Whether you're using pre-built LLMs like OpenAI's or models from Hugging Face, a solid grasp of data is key to ensuring your AI agent performs correctly. You need to understand the data before you can expect your agent to work effectively.

Checkout video for more detailed :https://youtu.be/zppJSAIbDg0?si=iM4L_BeAhqaeW9aU

Pre-trained Ai models in huggingface

Saad Ali — Sun, 09 Feb 2025 02:40:22 +0000

Been diving into Hugging Face lately and it's seriously impressive! Their transformer models are a huge help, especially when building AI apps. Think about it: instead of manually coding all the LLM integrations for your AI agent, you can use these pre-built transformer models from Hugging Face. Saves a ton of time and makes the whole process way smoother. Definitely worth checking out if you're working with AI

you can download and integrate all type of models, for example

text to text

text to image

image to video

text to video

it's really cool instead of implement from scratch use these models

they support all libraries and frameworks

. #AI #MachineLearning #HuggingFace #Transformers

Run DeepSeek R1 in your local machine

Saad Ali — Thu, 06 Feb 2025 00:52:22 +0000

Run DeepSeek-R1 Locally simple

Step 1: Install Ollama if you'r using ubuntu

command : curl -fsSL https://ollama.com/install.sh | sh

Step 2: Download and run DeepSeek-R1

run,

command :ollama run deepseek-r1

and

replace 'x' with parameter size you want (1.5b, 7b, 8b, 14b, 32b, 70b, 671b):

command: ollama run deepseek-r1:Xb

Step 3: Running DeepSeek-R1 in the background

command : ollama serve

now deepseek starts running on your local machine terminal if still not working then again enter this command

command : ollama run deepseek-r1

now you start chat with deepseek r1 in your local machine terminal..

nest (unit,integration,e2e) testing techniques and concepts

Saad Ali — Mon, 03 Feb 2025 04:14:22 +0000

Today, I learned about testing in NestJS, including how to implement it in a structured way using multiple modules. I covered various types of testing, such as unit testing, integration testing, and end-to-end (e2e) testing.
I also explored testing TypeORM modules, as well as best practices for testing Prisma. Additionally, I looked into GraphQL testing techniques and best practices.
If you want to learn more about testing, visit my GitHub, where you can easily understand even complex techniques.

GitHub: https://github.com/Saad-04/nest-Advance-fundamental-concepts

Backend engineering fundamentals and concepts

Saad Ali — Mon, 03 Feb 2025 04:13:00 +0000

hi there, my name is Saad , these are basics concepts of whole backend engineering that every software developers need to understand instead of directly learning modern tool and framework if you understand these concepts then after pickup any framework language won't matter, but the only difference is all framework use different techniques and framework concepts but behind same engineering fundaments and concepts running. This blog is just for basic fast understanding, you need to explore more things in more detail by yours research.

PREREQUISITES

A high-level understanding
protocol(HTTPS,HTTP/2,HTTP/3,UDP,gRPC,FTP,WEBSOCKETS,TCP,QUIC,security layers(TLS,SSL) )
A high-level understanding
HTTP protocol
Routing
Serialisation and deserialisation
Authentication and authorization
Validation and transformation
Middlewares
Request context
Handlers, controllers and services
CRUD deepdive
RESTful architecture and best practices
Databases
Business logic layer (BLL)
Caching
Transactional emails
Task queuing and scheduling
Elasticsearch
Error handling
Config management
Logging, monitoring and observability
Graceful shutdown
Security
Scaling and performance
Concurrency and parallelism
Object storage and large files
Real-time backend systems
Testing and code quality
12 factor app(Codebase, Dependencies, Configuration, Backing Services, Build, Release, Run, Processes, Port Binding, Concurrency, Disposability, Dev/Prod Parity, Logs, Admin Processes.)
OpenAPI standards
Webhooks
Web Servers
How web servers work?
Dynamic vs Static Content
E-Tags
Relational vs NoSQL
ACID
Proxies (Reverse Proxies, Load balancer)
What is difference between Proxy vs Reverse Proxy?
Layer 7 Proxy vs Layer 4 Proxy
Reverse Proxy applications
Load Balancing algorithms
When to use Caching
Message queue, Pub/Sub
When to use PubSub messaging first queue.
Web Frameworks (API authoring)
Express, Django, Node JS
Message Formats (JSON, protobuf)
JSON & protobuf
Encryption,Firewall.

DevOps for backend engineers

docker
docker compose
ci/cd pipelines(jenkins)
deployments
orchestration plateform kubernetes

nestjs microservice using gprc rabbitMQ redis docker docker-compose enterprise level repo startar pack

Saad Ali — Fri, 31 Jan 2025 19:33:32 +0000

Today i 12 hours to have about nestjs-microservices how to setup and perform operation how to make a connection between different services using TCP or gRPC ,RabbitMQ, Kafka ,Redis in different independent microservices.
how we use packages inside a monorepo for configuration for the different services instead of write separate configuration for each service
you can checkup my github if want to understad and clone microservices for your project instead of writing from scratch everytime use my repo: https://github.com/Saad-04/nestjs-microservice-RabbitMQ-kafka-redis-grpc

these are these architecture of full project

I cover these topics today
setting up packages based nx monorepo on pnpm workspace
building TCP based services (one gateway and two TCP based services )
building gRPC based services (one gateway and two gRPC based services )
building kafka/rabbit MQ based services (one gateway and two event driven services )

ELK logging monitoring

Saad Ali — Thu, 30 Jan 2025 02:34:34 +0000

Loggin Monitoring with ELK many more…
Today, I have learned a very advanced topic about ELK. Basically, what logging is, the best practices for managing an application's entire log, and how to use the Winston logging library to handle logging. I learned how to send log reports to Logstash, then Logstash sends logs to Elasticsearch, and finally, Elasticsearch sends them to Kibana for analysis. I also learned how to implement and integrate all these tools using Docker containerization. Additionally, I explored how to send logs directly to Elasticsearch using the Winston library instead of integrating Logstash to handle logs.
So today, I learned these topics by spending about 12 hours a day. In a few days, I will be building some software systems to apply all these concepts and techniques in my project.

this some basic flow how architecture look like , this is app api level logging integration

this is the full flow of logging techniques how look like at the end in kibana ui ->

Understanding Logging in NestJS: Learn about the built-in logging capabilities of NestJS and the importance of effective logging.
Introducing the ELK Stack: Discover the components of the ELK Stack (Elasticsearch, Logstash, Kibana) and their benefits for logging.
Setting Up the ELK Stack: Install and configure the ELK Stack components for your development environment.
Integrating NestJS with Logstash: Send NestJS logs to Logstash for centralized collection.
Indexing Logs in Elasticsearch: Index your NestJS logs in Elasticsearch for efficient search and analysis.
Visualizing Logs with Kibana: Use Kibana to explore, search, and visualize your NestJS logs.
Advanced Logging Techniques: Explore advanced techniques like structured logging, correlation IDs, and log rotation.
Best Practices for Logging: Follow best practices for effective logging in NestJS applications.

Benefits of Using ELK for NestJS Logging:
Centralized Logging: Collect logs from all your NestJS applications in a single location.
Powerful Search and Analysis: Use Elasticsearch's advanced search capabilities to analyze your logs.
Real-time Monitoring: Monitor your NestJS applications in real-time using Kibana dashboards.
Scalability: The ELK Stack can handle large volumes of logs.

mind set

Saad Ali — Thu, 30 Jan 2025 01:19:38 +0000

"The more we value things outside our control, the less control we have."

Engineering concept

Saad Ali — Wed, 29 Jan 2025 03:16:59 +0000

Today, I completed learning some NestJS framework concepts by spending about 13 hours. I already knew many of these things, as I have used them multiple times in Node.js. However, these were some high-level advanced topics with practical implementation. I focused on the best actual practices for building high-quality software systems, whether for startups or other purposes. I learned this because, in a few days, I will be building a large-scale backend system using multiple tools and the latest high-performance technologies. I have a complete architecture plan, especially in NestJS, which I will showcase soon.

1 Building Dynamic Nest JS Modules.
2 Core concepts of nest js Baseline code + PNPM with Nx + Github [Introduction to a monorepo with and Yarn workspaces]
basically a single work space for handling multiple projects and apps in a single place and i also use nx tool to speed up development instead of manually time consuming REST APIs (basic Building blocks )
5 Modules, Services, DI framework [Nest JS DI Framework]high level understanding
6 Setting up a PostgreSQL database with TypeORM(just single postgress setup package for multiple projects instead of create databases for each app )
7 API with NestJS Creating dynamic modules as monorepo package()
8 API with NestJS Defining dynamic modules
9-Dockerizing ,docker-compose to run multiple container in Nest JS Application using pmpn workspace speed up by nx
10- Advance dependency injection and their techniques, resolve cyclic dependency
11-dynamic modules using packages like separate from actual app using dependency injection on pnpm workspace
12-global dynamic module to inject into other app via pnpm packages cover advance custom provider techniques like class-based, non-class bases providers , useclass,useExisting, usevalue providers and technique to use all of them in single class by adding through global dynamic module
13- Error handling techniques and api integration in perfect way, i use openapi(swagger) for testing,

you can checkup first three folders for that inside the app director really helpfull for you if you want to diving into the nestjs core: https://github.com/Saad-04/nestjs-advanced-topics

such a nice journy so in tomarrow fully try to complete whole all fundamental of nestjs core like advance fundamentals and technique by striving and persevering , after understand i will show you some project by that really exited

if you already know nest js and want to learn advance best software practices then checkout this playlist:https://www.youtube.com/playlist?list=PLIGDNOJWiL1_YrquGbDDN4BNNRy678Spr