DEV Community: Louai Boumediene

Webhook Service: How To Approach System Design Interview + Practical Example

Louai Boumediene — Fri, 25 Apr 2025 17:15:25 +0000

This article was originally published on My Website

Introduction

System design is how you plan before you build. Not just code, but full picture: features, traffic, failures, scale and security. When you grow in level, company want you to think more, not just do more. Junior build, senior design before build. That's why system design interview show how senior you think.

🔥 Motivation

Recently I've been in a system design interview where, despite having most of the technical knowledge covered like queues, load balancers, API gateways, workers it wasn't the best experience due to lack of proper structure in answering system design questions. So, instead regretting the past, let's ensure better future experiences!

How To Approach a System Design Interview

💡 Tip: Interviewer will definitely maintain an atmosphere of ambiguity and open ended scope, so instead of being confused, embrace it to show how well your thinking process is, and how you deal with problems as a true software engineer

Step 1: Understand the Problem

First thing, don't rush. Listen careful to what the interviewer is asking for. Ask questions back if something is not clear. Make sure you 100% understand what you need to build before you start talking.

Step 2: Define & Split Requirements To Two Parts

Functional Requirements: State what the system should do in clear separate bullet points.
Non Functional Requirements: How well the system do what it should do (Scalability, Security, Fault-tolerance). Communicate them well, so the interviewer see you are thinking in full picture.

Step 3: Address Functional Requirements & High Level Design

Draw the big pieces. Like: client, server, database, cache, queue, load balancer.
Explain how they talk to each other. No need to go deep yet, just show that you know how to build a full system.

Step 4: Address Non Functional Requirements & Low Level Design

Walk through each and every part of the system and state how it can potentially fail and how you are planning to address or prevent that specific component's failure from failing the whole system.
Pick the potential bottlenecks in the system and discuss how u can scale them, like database by sharding it, or the consumer worker by spinning more replicas or the message queue by configuring it to write jobs to disk so in case of failure or restart it resumes from where it left.
Finally highlight potential security risks with the current system and address them.

💡 Tip: It would be really beneficial especially at big companies if you can do some calculations regarding memory, throughput, how many requests per second the system can handle and those stuff using techniques like Back of the envelope calculations

Practical Example: Design a Webhook Service

Step 1: Understand the Problem

Webhooks are way to get real-time updates from other systems. Unlike normal APIs where we keep asking: Any updates?... yooo anything new?.... webhooks push data to us when something happens. For example, Stripe sends us webhook when payment done, Shopify when order ships, GitHub when code pushed.

Step 2: Define & Split Requirements

Functional Requirements:

Accept webhook events from external systems
Process these events and do operations based on them
Save original event data and processing results for tracking

Non-Functional Requirements:

High availability: system must work even when parts fail
Security: Prevent webhook URL ID from being stolen, Rate limiting
At-least-once processing: each event must be processed at least once
Scalability: Can handle millions of requests per day

Step 3: Address Functional Requirements & High Level Design

Let's start with basic design. When external system sends webhook event, our service needs to:

Have API endpoint to receive event
Process it
Save it to database

Basic design looks like:

But this design has problem: if handler fails after processing but before saving to database, we lose the event.

Better design with message queue:

How this works:

External system sends event to our API
Request Handler accepts event and puts in message queue
Request Handler sends 200 OK response
Queue Consumer takes event from queue
Queue Consumer processes event
Queue Consumer saves result to database
Once saved, event removed from queue

Step 4: Address Non-Functional Requirements & Low Level Design

Handling Failures

Request Handler Failures:

If handler fails before putting event in queue: External system won't get 200 OK, so will retry
Use timeouts and circuit breakers to prevent hanging

Message Queue Failures:

Use durable queues that save messages to disk
Set up queue on multiple servers so if one fails, others work

Queue Consumer Failures:

Run multiple consumer instances
Only remove message from queue after successful database save
Auto-restart failed consumers

Database Failures:

Retry database writes with increasing delays
Set up database replication

Security Concerns

HMAC Signatures:
- External system and our service share secret key
- External system sends request with HMAC hash of payload
- Our service calculates same hash and compares
- If match, we know request is real
IP Whitelisting:
- Only accept requests from known IP addresses
Rate Limiting:
- Limit requests from same client (like 50 per minute)
- Prevents attacks that send too many requests (Dos attacks)

Handling Duplicate and Out-of-Order Requests

For Duplicates:

Use idempotency keys: store unique ID for each event
Before processing, check if already seen this ID
Some message queues have built-in deduplication

For Out-of-Order Events:

Don't assume events arrive in correct order
Make processing logic handle any order
Example: if "invoice.paid" arrives before "invoice.created"
- Get latest data from source API
- Use timestamps to know which event is newer
- Skip processing outdated events

By following these steps, we create webhook service that's reliable, secure, and handles real-world problems like failures, duplicates, and out-of-order events.

Acknowledgment

Sleek & Minimal Developer Portfolio Website

Louai Boumediene — Tue, 25 Mar 2025 20:13:46 +0000

Overview

This is a sleek, modern, and highly customizable portfolio template built using Astro. Designed with performance, accessibility, and SEO in mind, it offers a clean and minimalistic experience while providing all the essential features needed for a professional online presence.

Live Site

Check it out live here

Features

🌙 Light & Dark Mode – Seamlessly switch between themes.
📱 Mobile-Friendly – Fully responsive design.
📖 Blog Section – Supports MDX with automatic table of contents (TOC) and pagination.
🏷️ Categorized Articles – Filter posts by series or category.
🚀 Projects Showcase – Detailed overview of projects with images and descriptions.
🖼️ SEO & Open Graph – Optimized metadata and dynamic OG images.
🎨 Sleek & Modern Design – Crafted with a focus on simplicity and elegance.

Tech Stack

Built using the following technologies:

Astro – Blazing fast static site generation.
TailwindCSS – Highly customizable utility-first CSS framework.
TypeScript – Strict type safety for maintainability.
Prettier – Code formatting for consistency.
Framer Motion – Smooth animations and transitions.

How to Use

Customization is simple and straightforward:

All customizable data (name, links, content, etc.) is stored in lib/constants/ dir.
Images can be replaced in the public folder.
OG images for blogs and projects are stored in public/og/.
Theming follows shadcn's color palette and can be easily tweaked.

Contribute & Support

Love the project? Give it a ⭐ on GitHub! Contributions are always welcome—whether it's fixing a bug, adding a new feature, or even correcting a small typo.

👉 Check out the repo and make your first pull request!

Built with ❤️ using Astro.

Mastering SQL: Comprehensive Guide To Structured Query Language

Louai Boumediene — Wed, 08 May 2024 13:27:18 +0000

Introduction

In the previous blog article we've explored together a comprehensive guide on:

Mastering Relational Database Design: A Comprehensive Guide

Louai Boumediene ・ May 6 '24

#webdev #learning #database #backend

an in this article, we will discover in a high level the syntax and core features of the language that is actually used to bring the database schema design to existence... SQL.

So... Let's dive right into it!

What is SQL

SQL, or Structured Query Language, is the facade language used to talk to all RDMSs, however for the most time as backend engineers (we're not database engineers) we don't use SQL directly, and we rather use Object Relational Mappers ORMs which provides a better more reliable interface on top of raw SQL to communicate with database in our preferred programming language, however we still need a basic knowledge of SQL for quick and simple manipulation and interactions with the database.

Basic SQL Queries

Basic SQL queries involve selecting specific data from a database table using commands like SELECT, DISTINCT, WHERE, LIMIT, and OFFSET.

-- Selecting all columns from a table
SELECT * FROM employees;

-- Selecting specific columns
SELECT first_name, last_name FROM employees;

-- Using DISTINCT to get unique values
SELECT DISTINCT department_id FROM employees;

-- Using WHERE to filter results
SELECT * FROM employees WHERE department_id = 10;

-- Using LIMIT to limit the number of results
SELECT * FROM employees LIMIT 5;

-- Using OFFSET to skip certain rows
SELECT * FROM employees OFFSET 5;

Filtering Data

Filtering data in SQL allows you to retrieve specific records from a table based on certain conditions, such as using comparison operators like >, <, =, and logical operators like AND, OR, and NOT.

-- Using comparison operators
SELECT * FROM employees WHERE salary > 50000;

-- Using logical operators
SELECT * FROM employees WHERE department_id = 10 AND salary > 50000;

-- Using IN and NOT IN
SELECT * FROM employees WHERE department_id IN (10, 20);

-- Using BETWEEN
SELECT * FROM employees WHERE salary BETWEEN 40000 AND 60000;

-- Using LIKE for pattern matching
SELECT * FROM employees WHERE last_name LIKE 'S%';

Sorting Data

Sorting data in SQL arranges the retrieved records in either ascending or descending order based on specified columns using the ORDER BY clause.

-- Sorting data in ascending order
SELECT * FROM employees ORDER BY salary;

-- Sorting data in descending order
SELECT * FROM employees ORDER BY salary DESC;

-- Sorting by multiple columns
SELECT * FROM employees ORDER BY department_id, salary DESC;

Aggregate Functions

Aggregate functions in SQL perform calculations on a set of values and return a single value. Common aggregate functions include COUNT, SUM, AVG, MIN, and MAX.

-- Counting the number of rows
SELECT COUNT(*) FROM employees;

-- Calculating total salary
SELECT SUM(salary) FROM employees;

-- Finding average salary
SELECT AVG(salary) FROM employees;

-- Finding minimum salary
SELECT MIN(salary) FROM employees;

-- Finding maximum salary
SELECT MAX(salary) FROM employees;

Grouping Data

Grouping data in SQL allows you to group rows that have the same values in specified columns using the GROUP BY clause, often used in conjunction with aggregate functions.

-- Grouping data by department
SELECT department_id, COUNT(*) FROM employees GROUP BY department_id;

-- Using HAVING to filter grouped data
SELECT department_id, COUNT(*) FROM employees GROUP BY department_id HAVING COUNT(*) > 5;

Joins

Joins in SQL combine data from multiple tables based on related columns to retrieve data that spans across those tables.

-- Inner Join
SELECT * FROM employees INNER JOIN departments ON employees.department_id = departments.department_id;

-- Left Join
SELECT * FROM employees LEFT JOIN departments ON employees.department_id = departments.department_id;

-- Right Join
SELECT * FROM employees RIGHT JOIN departments ON employees.department_id = departments.department_id;

-- Full Outer Join
SELECT * FROM employees FULL OUTER JOIN departments ON employees.department_id = departments.department_id;

Subqueries

Subqueries in SQL are nested queries within another query, used to retrieve data that depends on the result of another query.

-- Subquery example
SELECT * FROM employees WHERE department_id IN (SELECT department_id FROM departments WHERE location_id = 1700);

-- Correlated subquery example
SELECT * FROM employees e WHERE salary > (SELECT AVG(salary) FROM employees WHERE department_id = e.department_id);

Views

Views in SQL are virtual tables generated from the result of a query, providing a way to simplify complex queries and restrict access to certain data.

-- Creating a view
CREATE VIEW high_paid_employees AS SELECT * FROM employees WHERE salary > 80000;

-- Updating a view
CREATE OR REPLACE VIEW high_paid_employees AS SELECT * FROM employees WHERE salary > 90000;

-- Dropping a view
DROP VIEW IF EXISTS high_paid_employees;

Indexing

Indexing in SQL improves the performance of queries by creating indexes on columns, allowing for faster data retrieval.

-- Creating an index
CREATE INDEX idx_lastname ON employees(last_name);

-- Dropping an index
DROP INDEX idx_lastname;

Transactions

Transactions in SQL ensure data integrity by grouping SQL statements into atomic units, ensuring that either all statements are successfully executed or none of them are.

-- Beginning a transaction
BEGIN TRANSACTION;

-- Committing a transaction
COMMIT;

-- Rolling back a transaction
ROLLBACK;

Stored Procedures

Stored procedures in SQL are precompiled SQL code stored in the database that can be executed with a single command, often used to encapsulate frequently executed tasks

-- Creating a stored procedure
CREATE PROCEDURE get_employee (IN employee_id INT)
BEGIN
    SELECT * FROM employees WHERE employee_id = employee_id;
END;

-- Executing a stored procedure
CALL get_employee(100);

-- Modifying a stored procedure
ALTER PROCEDURE get_employee (IN employee_id INT)
BEGIN
    SELECT employee_id, first_name, last_name FROM employees WHERE employee_id = employee_id;
END;

-- Dropping a stored procedure
DROP PROCEDURE IF EXISTS get_employee;

Backup and Recovery

Backup and recovery in SQL involves creating backups of databases to protect against data loss and restoring them in case of database failure or corruption.

-- Creating a full backup
BACKUP DATABASE dbname TO disk = 'path_to_backup';

-- Creating a differential backup
BACKUP DATABASE dbname TO disk = 'path_to_backup' WITH DIFFERENTIAL;

-- Creating a transaction log backup
BACKUP LOG dbname TO disk = 'path_to_backup';

-- Restoring from a backup
RESTORE DATABASE dbname FROM disk = 'path_to_backup';

SQL Dialects and Vendor-Specific Extensions

While SQL is a standardized language, different database vendors have implemented their own extensions and dialects, leading to slight variations in how SQL is written and executed across different RDBMSs. These variations can impact the syntax, functions, and features available to developers working with specific database systems.

For example, here are a few examples of vendor-specific SQL dialects and extensions:

Oracle SQL:

Oracle SQL includes proprietary extensions like hierarchical queries, analytic functions, and the MODEL clause for data modeling.
Example: SELECT CUBE(product, region) FROM sales; (CUBE is an Oracle-specific extension for generating subtotals)

SQL Server (Microsoft):

SQL Server includes T-SQL (Transact-SQL), which extends standard SQL with features like control-of-flow language constructs, error handling, and procedural programming constructs.
Example: SELECT ISNULL(column_name, 'default_value') FROM table_name; (ISNULL is a T-SQL function)

MySQL:

MySQL includes extensions like spatial data types and functions, window functions, and the REGEX operator for regular expression matching.
Example: SELECT column_name REGEXP '^pattern' FROM table_name; (REGEXP is a MySQL-specific operator)

PostgreSQL:

PostgreSQL includes extensions like arrays, JSON support, full-text search, and range data types.
Example: SELECT column_name || 'suffix' FROM table_name; (|| is a PostgreSQL-specific string concatenation operator)

So the most logical way to go about is:

The Basics: Know the basics of Relational Dabase Design: Entitnes, keys, Relationships, …. .
SQL: Learn the fundamentals SQL as a standalone lanugage.
Pick RDBMs: Pick one RDBMS that sweets you the most, and simply use it adhearing to all its special syntax and conspets.

Conclusion

SQL is a vast field in itself, encompassing a multitude of concepts beyond what we've touched upon. Transactions, backups, and file management are just a few examples of the broader scope. However, for backend engineers like us, the fundamentals we've covered suffice as a foundational overview of SQL.

Mastering Relational Database Design: A Comprehensive Guide

Louai Boumediene — Mon, 06 May 2024 21:49:16 +0000

Introduction

In today's data-driven world, storing and managing information efficiently is crucial for businesses and organizations of all sizes. Relational databases have emerged as a powerful solution for organizing and manipulating data in a structured and scalable manner. In this blog post, we'll explore the fundamentals of relational databases, their management systems, and the principles that underpin effective database design.

What is a Database?

A database is a structured collection of data that is organized and stored in a way that facilitates efficient retrieval, manipulation, and management. Think of it as a digital filing cabinet, where instead of physical folders and documents, you have tables and records neatly organized for easy access.

What is a Relational Database?

A relational database is a type of database that organizes data into tables (relations) with rows (records) and columns (fields). These tables are interconnected through relationships, allowing data to be accessed and combined in various ways. Imagine a collection of spreadsheets, each representing a different aspect of your data, but with the ability to link and combine information across them seamlessly.

RDBMS

A Relational Database Management System (RDBMS) is a software application designed to create, manage, and interact with relational databases. It provides a structured framework for storing, retrieving, and manipulating data within the database. Some popular examples of RDBMS include MySQL, PostgreSQL, Oracle, and Microsoft SQL Server.

Introduction to SQL

SQL (Structured Query Language) is the standard programming language used to interact with relational databases. It allows you to create, read, update, and delete data within the database, as well as define and modify the database structure itself. SQL is like a universal language that enables you to communicate with different RDBMS platforms.

Naming Conventions

In SQL, following consistent naming conventions is crucial for clarity and maintainability. Here's an example:

-- Good naming conventions
CREATE TABLE customers (
   customer_id INT PRIMARY KEY,
   first_name VARCHAR(100),
   last_name VARCHAR(100),
   email VARCHAR(100)
);

What is Database Design?

Database design is the process of creating an efficient and organized structure for storing and managing data in a database. It involves defining tables, columns, relationships, and constraints to ensure data integrity, minimize redundancy, and optimize performance. Proper database design is the foundation for building robust and scalable applications.

Data Integrity

Data integrity refers to the accuracy, consistency, and reliability of data stored in a database. It ensures that the data follows specific rules and constraints, preventing errors and inconsistencies. There are three types of data integrity:

Entity Integrity: Ensures each row in a table is uniquely identifiable by a primary key, and the primary key cannot have null values.
Referential Integrity: Maintains relationships between tables by ensuring foreign key values in one table match the primary key values in another table.
Domain Integrity: Enforces valid entries for a given column by restricting the data type, format, and range of values that can be stored.

-- Example: Enforcing data integrity
CREATE TABLE orders (
    order_id INT PRIMARY KEY, -- Entity integrity
    customer_id INT FOREIGN KEY REFERENCES customers(customer_id), -- Referential integrity
    order_date DATE NOT NULL, -- Domain integrity
    total_amount DECIMAL(10, 2) CHECK (total_amount >= 0) -- Domain integrity
);

Database Terms

Table: A collection of related data organized in rows and columns.
Row: A single instance or entry in a table (also known as a record or tuple).
Column: A specific characteristic or property of the data in a table (also known as a field or attribute).
Primary Key: A column or combination of columns that uniquely identifies each row in a table.
Foreign Key: A column or combination of columns that references the primary key of another table, establishing a relationship between the two tables.
Join: An operation that combines rows from two or more tables based on a related column.
Index: A data structure that improves the performance of data retrieval operations by creating a sorted representation of the data in a table.
View: A virtual table that is dynamically generated from one or more underlying tables.
Stored Procedure: A pre-compiled collection of SQL statements that can be executed as a single unit.
Trigger: A special type of stored procedure that is automatically executed when a specific event occurs in a table, such as an INSERT, UPDATE, or DELETE statement.

Atomic Values

In database design, it's important to store atomic values, which means storing the smallest pieces of information that cannot be further divided. This principle helps maintain data integrity and avoid redundancy.

For example, instead of storing a customer's full name in a single column, it's better to separate it into first name and last name columns. This way, you can easily search, sort, or manipulate each part of the name independently.

-- Example: Storing atomic values
CREATE TABLE customers (
    customer_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50),
    email VARCHAR(100)
);

Introduction to Keys

Keys are essential components in database design that help ensure data integrity and establish relationships between tables. They serve as unique identifiers for records and enable efficient data retrieval and manipulation.

Primary Key Index

A primary key is a column or a combination of columns that uniquely identifies each record in a table. It ensures that each record is unique and can be easily located. Primary keys are typically indexed to improve query performance.

CREATE TABLE customers (
    customer_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50),
    email VARCHAR(100)
);

Look up Table

A lookup table, also known as a reference table or a code table, is a table that contains predefined sets of values that can be used to populate columns in other tables. Lookup tables help maintain data integrity by ensuring consistency and reducing redundancy.

-- Lookup table for product categories
CREATE TABLE product_categories (
    category_id INT PRIMARY KEY,
    category_name VARCHAR(100)
);

-- Products table referencing the lookup table
CREATE TABLE products (
    product_id INT PRIMARY KEY,
    product_name VARCHAR(100),
    category_id INT,
    FOREIGN KEY (category_id) REFERENCES product_categories(category_id)
);

Superkey and Candidate Key

A superkey is a set of one or more columns in a table that uniquely identifies each record. A candidate key is a minimal superkey, meaning it contains no unnecessary columns. In other words, a candidate key is a superkey with the minimal number of columns required to uniquely identify each record.

Primary Key and Alternate Key

A primary key is a candidate key chosen as the main unique identifier for a table. An alternate key, also known as a unique key, is any other candidate key that could have been chosen as the primary key but was not.

CREATE TABLE employees (
    employee_id INT PRIMARY KEY,
    email VARCHAR(100) UNIQUE, -- Alternate key
    first_name VARCHAR(50),
    last_name VARCHAR(50)
);

Surrogate Key and Natural Key

A surrogate key is an artificial key (often a sequential number or a GUID) that is used as the primary key in a table. It has no inherent meaning or relationship to the data itself. A natural key, on the other hand, is a key derived from the data itself, such as an employee ID or a product code.

-- Surrogate key
CREATE TABLE orders (
    order_id INT PRIMARY KEY IDENTITY(1,1), -- Surrogate key
    customer_name VARCHAR(100),
    order_date DATE
);

-- Natural key
CREATE TABLE products (
    product_code VARCHAR(10) PRIMARY KEY, -- Natural key
    product_name VARCHAR(100),
    price DECIMAL(10,2)
);

Should I use Surrogate Keys or Natural Keys?

The choice between surrogate keys and natural keys depends on several factors, including the nature of the data, the likelihood of data changes, and the potential for duplication or conflicts.

Surrogate keys are often preferred because they are:

Immutable: They do not change over time, even if the data changes.
Guaranteed unique: They are generated by the database system, ensuring uniqueness.
Opaque: They do not reveal any information about the data itself, which can be beneficial for security and privacy.

Natural keys, on the other hand, can be advantageous when:

The data has inherent uniqueness, such as product codes or employee IDs.
The data is unlikely to change over time, reducing the risk of conflicts or duplication.
There is a need for human-readable and meaningful identifiers.

Foreign Key

A foreign key is a column or a combination of columns in one table that references the primary key of another table. It establishes a link between the two tables and enforces referential integrity, ensuring that data in the child table is valid and consistent with the data in the parent table.

CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    customer_id INT,
    order_date DATE,
    FOREIGN KEY (customer_id) REFERENCES customers(customer_id)
);

NOT NULL Foreign Key

In some cases, it might be desirable to have a NOT NULL constraint on a foreign key column, meaning that the column cannot have a null value. This constraint ensures that every record in the child table is associated with a valid record in the parent table.

CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    customer_id INT NOT NULL,
    order_date DATE,
    FOREIGN KEY (customer_id) REFERENCES customers(customer_id)
);

Foreign Key Constraints

Foreign key constraints define the rules for referential integrity between tables. These constraints can include actions to be taken when a referenced record in the parent table is updated or deleted, such as:

CASCADE: When a record in the parent table is updated or deleted, the corresponding records in the child table are also updated or deleted.
SET NULL: When a record in the parent table is updated or deleted, the corresponding foreign key values in the child table are set to NULL.
NO ACTION: When a record in the parent table is updated or deleted, the corresponding foreign key values in the child table remain unchanged, and the operation is rolled back if it violates referential integrity.

CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    customer_id INT,
    order_date DATE,
    FOREIGN KEY (customer_id) REFERENCES customers(customer_id) ON UPDATE CASCADE ON DELETE SET NULL
);

Simple Key, Composite Key, Compound Key

A simple key is a single column used as a primary key or a foreign key.
A composite key is a combination of two or more columns used as a primary key or a foreign key.
A compound key is a combination of two or more simple keys used as a foreign key.

-- Simple key
CREATE TABLE orders (
    order_id INT PRIMARY KEY,
    ...
);

-- Composite key
CREATE TABLE order_items (
    order_id INT,
    product_id INT,
    quantity INT,
    PRIMARY KEY (order_id, product_id)
);

-- Compound key
CREATE TABLE shipments (
    shipment_id INT PRIMARY KEY,
    order_id INT,
    product_id INT,
    FOREIGN KEY (order_id, product_id) REFERENCES order_items(order_id, product_id)
);

Relationships

Relationships are the cornerstone of relational databases, allowing you to connect and combine data from different tables. There are three main types of relationships:

One-to-One Relationships

In a one-to-one relationship, each record in one table is associated with exactly one record in another table, and vice versa. For example, consider a database where each employee has one and only one manager, and each manager manages one and only one employee. This type of relationship is relatively rare in practice.

One-to-Many Relationships

In a one-to-many relationship, each record in one table (the "one" side) can be associated with multiple records in another table (the "many" side). For example, in a database for a school, one teacher can teach multiple classes, but each class is taught by only one teacher.

Many-to-Many Relationships

In a many-to-many relationship, each record in one table can be associated with multiple records in another table, and vice versa. For example, in a database for a university, a student can enroll in multiple courses, and each course can have multiple students enrolled.

Summary of Relationships

One-to-One: One record in Table A is related to one and only one record in Table B, and vice versa.
One-to-Many: One record in Table A can be related to multiple records in Table B, but one record in Table B can be related to only one record in Table A.
Many-to-Many: Multiple records in Table A can be related to multiple records in Table B, and vice versa.

Designing Relationships (SQL Implementation)

1 - Designing One-to-One Relationships

To design a one-to-one relationship, you can either include all the columns from both tables in a single table or create two separate tables and use a foreign key constraint to link them.

-- Option 1: Single table
CREATE TABLE employees (
    employee_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50),
    manager_first_name VARCHAR(50),
    manager_last_name VARCHAR(50)
);

-- Option 2: Two tables with foreign key
CREATE TABLE employees (
    employee_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50),
    manager_id INT UNIQUE,
    FOREIGN KEY (manager_id) REFERENCES employees(employee_id)
);

2 - Designing One-to-Many Relationships

To design a one-to-many relationship, you typically create two tables: a parent table (the "one" side) and a child table (the "many" side). The child table includes a foreign key column that references the primary key of the parent table.

-- Parent table
CREATE TABLE teachers (
    teacher_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50)
);

-- Child table
CREATE TABLE classes (
    class_id INT PRIMARY KEY,
    class_name VARCHAR(100),
    teacher_id INT,
    FOREIGN KEY (teacher_id) REFERENCES teachers(teacher_id)
);

3 - Designing Many-to-Many Relationships

To design a many-to-many relationship, you typically create a third table (called a junction table or associative table) that links the two main tables together. This junction table includes foreign key columns that reference the primary

-- Table 1
CREATE TABLE students (
    student_id INT PRIMARY KEY,
    first_name VARCHAR(50),
    last_name VARCHAR(50)
);

-- Table 2
CREATE TABLE courses (
    course_id INT PRIMARY KEY,
    course_name VARCHAR(100),
    description TEXT
);

-- Junction or Intemediary table
CREATE TABLE enrollments (
    enrollment_id INT PRIMARY KEY,
    student_id INT,
    course_id INT,
    FOREIGN KEY (student_id) REFERENCES students(student_id),
    FOREIGN KEY (course_id) REFERENCES courses(course_id)
);

Parent Tables and Child Tables

In a one-to-many or many-to-many relationship, the table on the "one" side is often referred to as the parent table, while the table on the "many" side is called the child table. The child table contains a foreign key that references the primary key of the parent table.

For example, in the teacher-class relationship, the teachers table is the parent table, and the classes table is the child table. Similarly, in the student-course relationship, the students and courses tables are parent tables, while the enrollm

Introduction to Entity Relationship Modeling

Entity Relationship Modeling (ER Modeling) is a technique used in database design to represent the logical structure of a database visually. It helps identify the entities (tables), attributes (columns), and relationships between them, making it easier to understand and communicate the database design.

ER diagrams consist of the following components:

Entities: Represented by rectangles, entities are the tables or objects in the database.
Attributes: Listed inside the entity rectangle, attributes are the columns or fields that describe the entity.
Relationships: Represented by lines connecting entities, relationships depict the associations between entities.

Cardinality

Cardinality defines the numerical relationship between two entities. It specifies the maximum number of instances of one entity that can be associated with a single instance of another entity. The most common cardinalities are:

One-to-One (1:1): One instance of Entity A can be associated with at most one instance of Entity B, and vice versa.
One-to-Many (1:N): One instance of Entity A can be associated with multiple instances of Entity B, but one instance of Entity B can be associated with only one instance of Entity A.
Many-to-Many (M:N): Multiple instances of Entity A can be associated with multiple instances of Entity B, and vice versa.

In ER diagrams, cardinality is represented using specific notation, such as a single line for one-to-one, a line with an arrowhead for one-to-many, and a line with arrowheads at both ends for many-to-many relationships.

Modality

Modality refers to whether the existence of an entity instance depends on its relationship with another entity. There are two types of modality:

Partial modality: The existence of an instance does not depend on its relationship with another entity. For example, a customer can exist without having any orders.
Total modality: The existence of an instance depends on its relationship with another entity. For example, an order item cannot exist without an order.

In ER diagrams, modality is represented using specific notation, such as a single bar for partial modality and a double bar for total modality.

Introduction to Database Normalization

Database normalization is the process of organizing data in a database to reduce redundancy, minimize data anomalies (insertion, update, and deletion anomalies), and improve data integrity. It involves breaking down a database into smaller tables and defining relationships between them based on specific rules or normal forms.

The primary goals of database normalization are:

Eliminating redundant data
Ensuring data integrity
Facilitating data manipulation and maintenance

There are several normal forms in database normalization, each building upon the previous one. The most commonly used normal forms are:

First Normal Form (1NF)
Second Normal Form (2NF)
Third Normal Form (3NF)

1NF (First Normal Form of Database Normalization)

The First Normal Form (1NF) is the most basic form of normalization. It states that an attribute (column) in a table must have atomic values, meaning each cell in the table should contain a single value, not a set of values.

For example, consider a table with a column named "PhoneNumbers" that stores multiple phone numbers for a customer. This violates 1NF because the column contains a set of values instead of a single value. To conform to 1NF, you would need to separate the phone numbers into individual columns or create a separate table for phone numbers.

-- Violates 1NF
CREATE TABLE customers (
    customer_id INT PRIMARY KEY,
    name VARCHAR(100),
    phone_numbers VARCHAR(200) -- Stores multiple phone numbers, violating 1NF
);

-- Conforms to 1NF
CREATE TABLE customers (
    customer_id INT PRIMARY KEY,
    name VARCHAR(100),
    phone1 VARCHAR(20),
    phone2 VARCHAR(20),
    phone3 VARCHAR(20)
);

2NF (Second Normal Form of Database Normalization)

The Second Normal Form (2NF) builds upon 1NF by addressing the issue of partial dependencies. A table is in 2NF if it is in 1NF and every non-prime attribute (column) is fully dependent on the entire primary key.

In other words, if a table has a composite primary key (consisting of multiple columns), then all non-key columns must depend on the entire primary key, not just a part of it.

For example, consider a table with a composite primary key of (student_id, course_id) and a column grade. If the grade column depends only on the course_id and not on the combination of student_id and course_id, then the table violates 2NF.

-- Violates 2NF
CREATE TABLE student_courses (
    student_id INT,
    course_id INT,
    course_name VARCHAR(100), -- Depends only on course_id, not the entire primary key
    grade CHAR(2), 
    PRIMARY KEY (student_id, course_id)
);

-- Conforms to 2NF
CREATE TABLE student_courses (
    student_id INT,
    course_id INT,
    grade CHAR(2),
    PRIMARY KEY (student_id, course_id)
);

CREATE TABLE courses (
    course_id INT PRIMARY KEY,
    course_name VARCHAR(100)
);

3NF (Third Normal Form of Database Normalization)

The Third Normal Form (3NF) builds upon 2NF by addressing the issue of transitive dependencies. A table is in 3NF if it is in 2NF and every non-prime attribute is non-transitively dependent on the primary key.

In other words, if a non-key column is dependent on another non-key column, then the table violates 3NF, and the non-key columns should be separated into their own table.

For example, consider a table with columns student_id, student_name, class_id, and class_name. The student_name column depends on the student_id, and the class_name column depends on the class_id. However, the class_name column also transitively depends on the student_id through the class_id column. This violates 3NF.

-- Violates 3NF
CREATE TABLE student_classes (
    student_id INT,
    student_name VARCHAR(100),
    class_id INT,
    class_name VARCHAR(100),
    PRIMARY KEY (student_id, class_id)
);

-- Conforms to 3NF
CREATE TABLE students (
    student_id INT PRIMARY KEY,
    student_name VARCHAR(100)
);

CREATE TABLE classes (
    class_id INT PRIMARY KEY,
    class_name VARCHAR(100)
);

CREATE TABLE student_classes (
    student_id INT,
    class_id INT,
    PRIMARY KEY (student_id, class_id),
    FOREIGN KEY (student_id) REFERENCES students(student_id),
    FOREIGN KEY (class_id) REFERENCES classes(class_id)
);

By following the principles of database normalization, you can create well-structured and efficient databases that minimize redundancy, maintain data integrity, and facilitate data manipulation and maintenance.

Indexes (Clustered, Nonclustered, Composite Index)

Indexes are data structures that improve the performance of data retrieval operations in a database. They create a sorted representation of the data in a table, allowing for faster searches and queries. There are several types of indexes:

Clustered Index: A clustered index physically reorders the rows in a table based on the index key values. Each table can have only one clustered index.
Nonclustered Index: A nonclustered index is a separate object that contains the index key values and pointers to the corresponding rows in the table. A table can have multiple nonclustered indexes.
Composite Index: A composite index is an index that includes multiple columns in the index key. It can be either clustered or nonclustered.

-- Clustered index
CREATE CLUSTERED INDEX idx_customers_name
ON customers (last_name, first_name);

-- Nonclustered index
CREATE NONCLUSTERED INDEX idx_orders_date
ON orders (order_date);

-- Composite index
CREATE INDEX idx_products_category_price
ON products (category_id, price);

Data Types

In the world of databases, data types are like different shapes of containers that hold specific types of information. Just as you wouldn't store liquids in a basket or solid objects in a jar, databases need to enforce specific data types to ensure data integrity and consistency.

Some common data types in SQL include:

INT or INTEGER: Stores whole numbers, like 42 or 17.
FLOAT or DOUBLE: Stores decimal numbers, like 3.14159 or 0.00005.
VARCHAR or TEXT: Stores text data, like names or descriptions.
DATE or DATETIME: Stores date and time values, like '2023-05-06' or '2024-01-01 12:34:56'.
BOOLEAN: Stores true/false values, like 1 (true) or 0 (false).

Choosing the right data type is crucial because it affects how the data is stored, queried, and manipulated. For example, trying to store a large string in an INT column would result in an error or data truncation.

CREATE TABLE users (
    id INT PRIMARY KEY,
    name VARCHAR(50) NOT NULL,
    age INT,
    is_active BOOLEAN DEFAULT 1
);

In this example, we create a users table with columns for id (integer), name (string of up to 50 characters), age (integer), and is_active (boolean, with a default value of 1 or true).

Introduction to Joins

Joins are like bridges that connect different tables in a database, allowing you to combine and retrieve related data from multiple sources. They are a fundamental concept in relational databases and are essential for querying and manipulating data efficiently.

Inner Join

An inner join is like a friendly handshake between two tables, where only the rows that have matching values in both tables are included in the result set. It's a way to combine data from multiple tables based on a common column or set of columns.

SELECT users.name, orders.order_date
FROM users
INNER JOIN orders ON users.id = orders.user_id;

In this example, we retrieve the name column from the users table and the order_date column from the orders table, but only for rows where the id in the users table matches the user_id in the orders table.

Inner Join on 3 Tables (Example)

Let's say we have three tables: users, orders, and products. We want to retrieve the user's name, the order date, and the product name for each order. We can achieve this by performing an inner join across all three tables:

SELECT users.name, orders.order_date, products.product_name
FROM users
INNER JOIN orders ON users.id = orders.user_id
INNER JOIN products ON orders.product_id = products.id;

Here, we first join the users and orders tables on the id and user_id columns. Then, we join the result of that join with the products table on the product_id and id columns. This way, we can retrieve data from all three tables in a single query, but only for rows where the join conditions are met.

Introduction to Outer Joins

While inner joins are like friendly handshakes, outer joins are more like welcoming hugs. They include not only the matching rows from both tables but also the non-matching rows from one or both tables, depending on the type of outer join.

Right Outer Join

A right outer join is like a warm hug from the right table to the left table. It includes all rows from the right table, along with the matching rows from the left table. If there are no matching rows in the left table, the result will contain NULL values for the left table's columns.

SELECT users.name, orders.order_date
FROM users
RIGHT OUTER JOIN orders ON users.id = orders.user_id;

In this example, we retrieve all rows from the orders table (the right table), along with the matching name values from the users table (the left table). If an order doesn't have a matching user, the name column will contain NULL.

JOIN with NOT NULL Columns

Sometimes, you may want to perform a join only on columns that are not null. This can be useful when you want to exclude rows with missing data from the result set.

SELECT users.name, orders.order_date
FROM users
INNER JOIN orders ON users.id = orders.user_id AND users.name IS NOT NULL;

In this example, we perform an inner join between the users and orders tables, but we add an additional condition users.name IS NOT NULL to ensure that only rows with non-null name values are included in the result set.

Outer Join Across 3 Tables

Similar to the inner join example, we can perform outer joins across multiple tables. Let's say we want to retrieve all orders, along with the user's name and the product name, even if there are missing values in the users or products tables.

SELECT users.name, orders.order_date, products.product_name
FROM orders
LEFT OUTER JOIN users ON orders.user_id = users.id
LEFT OUTER JOIN products ON orders.product_id = products.id;

Here, we start with the orders table and perform a left outer join with both the users and products tables. This ensures that all orders are included in the result set, along with the matching user names and product names if available. If there are no matching rows in the users or products tables, the respective columns will contain NULL values.

Alias

Aliases are like nicknames for tables or columns in SQL queries. They can make queries more readable and easier to understand, especially when dealing with long table or column names, or when referencing the same table multiple times in a query.

SELECT u.name, o.order_date, p.product_name
FROM users u
INNER JOIN orders o ON u.id = o.user_id
INNER JOIN products p ON o.product_id = p.id;

In this example, we use the aliases u for the users table, o for the orders table, and p for the products table. This makes the query more concise and easier to read, without having to repeat the full table names multiple times.

Self Join

A self join is like a table having a conversation with itself. It's a way to join a table with itself, based on a specific condition or relationship within the same table. This can be useful when dealing with hierarchical or recursive data structures, such as employee-manager relationships or nested categories.

SELECT e.name AS employee, m.name AS manager
FROM employees e
LEFT OUTER JOIN employees m ON e.manager_id = m.id;

In this example, we perform a self join on the employees table to retrieve the name of each employee and their corresponding manager's name. We use a left outer join to ensure that all employees are included in the result set, even if they don't have a manager assigned.

Database Markup Language (DBML)

Database Modeling Language (DBML) is a simple and intuitive markup language for describing the structure of relational databases. It provides a human-readable way to define tables, columns, relationships, and constraints, making it easy to communicate and collaborate on database designs.

Getting Started with DBML

To get started with DBML, you'll need a text editor and a basic understanding of database concepts. Let's create our first DBML file:

// my_database.dbml

Table users {
  id int [pk, increment]
  username varchar
  email varchar [unique]
  created_at datetime [default: `now()`]
}

In this example, we've defined a users table with columns for id, username, email, and created_at. The [pk] tag specifies that id is the primary key, [increment] indicates auto-incrementing, [unique] ensures uniqueness for the email, and [default: now()] sets the default value of created_at to the current timestamp.

Creating Tables

DBML allows you to define multiple tables and their columns in a single file. Let's add more tables to our database:

// my_database.dbml

Table users {
  id int [pk, increment]
  username varchar
  email varchar [unique]
  created_at datetime [default: `now()`]
}

Table posts {
  id int [pk, increment]
  title varchar
  content text
  user_id int [ref: > users.id]
  created_at datetime [default: `now()`]
}

In this example, we've added a posts table with columns for id, title, content, user_id, and created_at. The [ref: > users.id] tag establishes a foreign key relationship between the user_id column in the posts table and the id column in the users table.

Defining Relationships

DBML supports various types of relationships between tables, including one-to-one, one-to-many, and many-to-many. Let's define some relationships in our database:

// my_database.dbml

Table users {
  id int [pk, increment]
  username varchar
  email varchar [unique]
  created_at datetime [default: `now()`]
}

Table posts {
  id int [pk, increment]
  title varchar
  content text
  user_id int [ref: > users.id]
  created_at datetime [default: `now()`]
}

Ref: users.id < posts.user_id

In this example, we've defined a one-to-many relationship between the users and posts tables. The Ref: users.id < posts.user_id line specifies that the id column in the users table is referenced by the user_id column in the posts table.

Adding Constraints

Constraints ensure data integrity and enforce rules on the database. DBML supports various constraints such as primary keys, foreign keys, unique constraints, and default values. Let's add some constraints to our tables:

// my_database.dbml

Table users {
  id int [pk, increment]
  username varchar [unique]
  email varchar [unique]
  created_at datetime [default: `now()`]
}

Table posts {
  id int [pk, increment]
  title varchar
  content text
  user_id int [ref: > users.id]
  created_at datetime [default: `now()`]
}

Ref: users.id < posts.user_id

In this updated example, we've added a [unique] constraint to the username column in the users table to ensure that each username is unique.

Documenting Your Database

DBML allows you to add comments and annotations to your database schema, making it easier to understand and maintain. Let's document our tables with comments:

// my_database.dbml

Table users {
  id int [pk, increment] // Unique identifier for users
  username varchar [unique] // User's username
  email varchar [unique] // User's email address
  created_at datetime [default: `now()`] // Date and time when the user was created
}

Table posts {
  id int [pk, increment] // Unique identifier for posts
  title varchar // Title of the post
  content text // Content of the post
  user_id int [ref: > users.id] // ID of the user who created the post
  created_at datetime [default: `now()`] // Date and time when the post was created
}

Ref: users.id < posts.user_id // Relationship between users and posts

Benefits of DBML

Simple and human-readable syntax
Database-agnostic approach
Free visualization tool at dbdiagram.io
Consistent conventions for readability and maintainability
Extensive documentation and examples

Conclusion

In wrapping up, relational database design forms the backbone of efficient data organization in modern applications. Understanding its principles empowers you to create robust and scalable databases. Next up, we'll delve into SQL syntax, unlocking the power to interact with databases effectively. Stay tuned for our exploration into the world of SQL!

Let's Build a Learning Management System (LMS): Hands-on Learning Approach

Louai Boumediene — Fri, 03 May 2024 20:52:17 +0000

Introduction

It's been a while since I kick-started my backend engineering journey, and the plan was simply to take each major concept in the backend landscape and learn it then learn one or two of its application like for example, learning about databases then learning PostreSQL & Mongodb, for more details checkout this blog article I wrote, which talks about this point in great details:

Unveiling the Backbone: Dive Deep into Backend Engineering Essentials!

Louai Boumediene ・ Apr 11 '24

#webdev #backenddevelopment #learning #career

but essentially after a while, I realized that it's not the most optimum way to make it to the other side where senior backend engineers are! So let's discuss together why I'll switch to a more practical and hands-on learning approach and how I'll do it...

Limitation Of Previous Theoretical Learning Approach

First of all, backend engineering is simply building things! of course you need to learn the necessary tools and building blocks (web servers, protocols, proxies, ....) so you can build something with them in the first place! but for someone who already has put a foot in the software engineering world, its easier, funnier and more practical to learn by building projects rather than learning pure concepts that you already have at least a basic idea with, and can be further understood by utilizing them in a project you're building.

Second, in the realm of backend, some things aren't just learnable by taking courses or reading articles and documentation, things like architectural concepts like micro-services, webhooks or operational skills like CI/CD and project maintenance and a lot more, one of you may argue that there is plenty of courses, docs and articles that teaches those things in details, but for me such concepts are better learned by doing then use their learning resources as references, just like you do with cooking recipes you don't study the recipe a head of time then cook it later!, the same can be said about a lot of backend engineering knowledge:

Better learned by doing rather than reading

Finally and most important for me is me realizing that learning backend concepts piece by piece won't take you far, and that because those concepts are glued together in away that makes learning them separately doesn't make to much sense in the real world.

The Project We'll Build and It's Requirements

Choosing the right project to work on wasn't easy. I wanted something that's packed with features so I can utilize lots of different technologies and learn as much as possible. But most importantly, it had to be something I'm really passionate about and could work on for a long time.

My choices varied between:

E-Commerce backend system
Social Media backend system
Blogging website backend system
E-Learning platform or a Learning management system (LMS)

After weighing my options, I've decided to build a Learning Management System (LMS).

An LMS is a great project that will gimme room to implement a lot of core backend concepts: beside the obvious ones: Framework, Database, Authentication I can also implement a lot of other cool things like: splitting it into separate micro-services, implementing a messaging queue so the system is more robust and less coupled and a lot of other technical requirements.

The requirement of the (LMS) I wanna build are as follows:

1. User Roles

1.1 Teacher

Can create, manage, and publish courses.
Has access to a dashboard for course management and analytics.

1.2 Student

Can enroll in courses, view course content, and track progress.
Has access to a dashboard for viewing enrolled courses and progress.

2. Core Features

2.1 Course Management

Teachers can create and manage courses, including adding modules, lessons, quizzes, and assignments.
Course content can include text, images, videos, presentations, and other multimedia elements.
Teachers can set prerequisites, deadlines, and grading criteria for courses.

2.2 Student Enrollment

Students can browse the course marketplace and enroll in courses.
Enrollment process includes payment integration for paid courses.
Students receive notifications and updates on enrolled courses.

2.3 Dashboards

Teacher Dashboard:
- Overview of course analytics, including enrollment, completion rates, and student performance.
- Tools for managing course content, assignments, and communication with students.
Student Dashboard:
- List of enrolled courses with progress tracking.
- Access to course materials, assignments, and grades.

2.4 Communication

Integrated messaging system for communication between teachers and students.
Discussion forums for course-specific discussions and peer interaction.
Announcements and notifications for important updates and deadlines.

2.5 Assessment and Grading

Support for various assessment formats, including quizzes, assignments, and exams.
Automatic grading for objective assessments and manual grading for subjective assessments.
Gradebook for teachers to track student performance and provide feedback.

2.6 Course Marketplace

Platform for course creators to publish and sell their courses.
Course discovery features, including search, filters, and recommendations.
Revenue sharing model for course creators and platform fees for transactions.

3. Technical Requirements

3.1 Frontend
which is the thing that it is not my priority for now, I'll focus on implementing a big part if not all the backend logic before I think of the front end

3.2 Backend

Scalable and secure backend infrastructure hosted on cloud service.
RESTful, gRPC, websockets, GraphQL APIs (maybe only restfull, maybe two of them maybe all I'm not sure yet) for communication between frontend and backend components.
Micro-services architecture where each part of the bakcend system will have each own containerized independent service
Integration with third party services (payment gateways and storage buckets, ...)

3.3 Database

Relational database management system is must for such big system with tones of related entities.
May NoSQL database for storing more instructed data like courses content and chapters......
An in-memory data base for caching purposes.

4. Security and Compliance

4.1 User Authentication

Secure authentication and authorization mechanisms for user accounts.
Support for OAuth2 providers like google, github....

4.2 Data Protection

Encryption of sensitive data in transit and at rest.
Compliance with data protection regulations (e.g., GDPR, CCPA).

4.3 Access Control

Role-based access control (RBAC) to restrict access to sensitive features and functionalities.
Granular permissions management for admins, teachers, and students.

The Chosen Tech-stack and Why

Well well well, the choice of the tech stack will be highly subjective and constrained to the following 3 points:

Keep away from what I already know: where I'll try to get out of my comfort zone and use technologies that I'm not already familiar with (at the end of the day a lot of standardized technologies I'm not familiar with cuz I'm still kinda of a beginner)
Technology spread and popularity: This one is pretty straightforward, since I'm new and yet learning no need to choose some fancy rust based backend framework that only few hundred people know how to use on the world!
What I personally see sexy and hot: Cuz most of the time there is multiple choices that are very popular, and I'm not familiar with but I choose one over the other for no reason but cuz it sounds sexier!

therefor here is my tech-stack for this project:

Backend Framework:

NestJS: Main backend framework, providing a modular and scalable architecture for building server-side applications with TypeScript.

Database:

PostgreSQL: Main relational database management system (RDBMS) used for storing and managing structured data in the project.

Programming Language:

TypeScript: Language of choice enforced by NestJS, offering strong typing and modern features for enhanced developer productivity and code maintainability.

Message Broker:

RabbitMQ: Messaging broker utilized for asynchronous communication and event-driven architecture, facilitating scalable and decoupled backend services.

Cloud Hosting Provider:

AWS (Amazon Web Services): Cloud infrastructure platform used for hosting, deploying, and scaling the project's backend services and resources.

Payment Integration:

Stripe: Payment processing platform integrated into the project for handling online payments securely and efficiently.

Object Storage:

Amazon S3 (Simple Storage Service): Scalable cloud storage solution employed for storing and retrieving large files and media assets in the project.

Caching:

Redis: In-memory data store utilized for caching frequently accessed data and improving the performance and responsiveness of the application.

Containerization:

Docker: Containerization platform employed for packaging the project's applications and dependencies into lightweight and portable containers.

Container Orchestration:

Kubernetes (K8s): Container orchestration tool utilized for automating deployment, scaling, and management of containerized applications in a clustered environment.

And Who Knows What Else...

Conclusion

What I truly like about this new approach is that there is a lot of skills that I'll be forced to learn, things like Database Design, Operational skills like CI/CD and a lot more. And Concerning Java and DSA nothing changed, my DSA journey is still running and the next blog post will be either DSA Part 1 or Relational Database Design, so stay tuned and see yaaaaaaaaaaaaaaaaaaa

Java 101: Zero to Hero Course

Louai Boumediene — Sat, 27 Apr 2024 18:16:48 +0000

Introduction:

Your introduction is engaging and sets a friendly tone for the blog. Here's a corrected version with some minor adjustments for clarity:

Welcome to Java 101, where I'll take you on my personal journey of learning Java. In this comprehensive blog post, we'll cover everything you need to know about the Java language, from setting up your development environment to diving deep into object-oriented programming and exploring advanced concepts.

Java is a versatile and powerful programming language used in a wide range of applications, from building web applications to developing mobile apps and enterprise systems. That's why it holds a special place in my heart.

So, get ready to embark on this adventure because it will be one of the longest blog posts I've ever written. I'll structure it as follows:

Part 1: The Fundamentals (variables, data types, flow controls, logical operators, ...)
Part 2: Object-Oriented Programming (Classes and Objects, Inheritance, Polymorphism, Encapsulation, Abstraction, Interfaces, ...)
Part 3: Advanced Java Topics (Exceptions, Generics, Collections, Functional Programming, Concurrency, Multithreading)

Let's dive in and explore the world of Java together!

Section 1 | Language Fundamentals

Sure! Let's break down each part of data types in a simple and easy-to-understand way.

Primitive Data Types

Primitive data types are the most basic building blocks of data in Java. They represent single values with no internal structure. Here are some common primitive data types:

byte: Represents a 1 byte whole number in the range [-128, 127].
short: Represents a 2 bytes whole number in the range [-32,768, 32,767].
int: Represents a 4 bytes whole number in a wide range.
long: Represents an 8 bytes whole number in a very wide range.
float: Represents a 4 bytes floating-point number, suitable for approximate representations of real numbers.
double: Represents an 8 bytes floating-point number, providing higher precision compared to float.
boolean: Represents true or false values.
char: Represents single characters like 'A', 'b', '%'.

Let's see some examples:

byte age = 25;
short population = 32000;
int distance = 150000;
long globalPopulation = 7760000000L; // Note the 'L' suffix for long literals

float temperature = 20.5f; // Note the 'f' suffix for float literals
double height = 5.9;

boolean isSunny = true;
char grade = 'A';

In this section, we've provided an overview of the sizes and ranges of various primitive data types in Java, including examples demonstrating their usage. Additionally, we introduced the float data type for representing floating-point numbers with approximate precision.

Reference Data Types

Reference data types refer to objects in Java. Unlike primitive data types, they have complex structures and are stored in memory dynamically. Here are some examples of reference data types:

String: Represents a sequence of characters, like "Hello, World!".
Array: Represents a collection of elements of the same type.

Let's see how we can use them:

String name = "Alice";
int[] numbers = {1, 2, 3, 4, 5};

Here, name is a String that stores the name "Alice", and numbers is an array of integers.

Input Reading Techniques

In Java, we can read input from the user using the Scanner class. Here's how we can use it to read integers and strings:

import java.util.Scanner;

public class InputExample {
    public static void main(String[] args) {
        Scanner scanner = new Scanner(System.in);

        System.out.print("Enter your age: ");
        int age = scanner.nextInt();

        System.out.print("Enter your name: ");
        String name = scanner.next();

        System.out.println("Hello, " + name + "! You are " + age + " years old.");

        scanner.close(); // Don't forget to close the scanner to release resources.
    }
}

When you run this program, it will prompt you to enter your age and name. Then it will print a greeting message with your input values.

That's it! These are the basics of data types in Java, explained in a simple and easy-to-understand way. Feel free to experiment with these examples to deepen your understanding!

Absolutely! Let's break down each concept of control flow with code examples for better understanding:

Comparison Operators

Comparison operators are used to compare two values and return a boolean result. Here are some common comparison operators:

Equal to (==): Returns true if two values are equal.
Not equal to (!=): Returns true if two values are not equal.
Greater than (>): Returns true if the left operand is greater than the right operand.
Greater than or equal to (>=): Returns true if the left operand is greater than or equal to the right operand.
Less than (<): Returns true if the right operand is greater than the left operand.
Less than or equal to (<=): Returns true if the left operand is less than or equal to the right operand.

int x = 5;
int y = 10;

System.out.println(x == y);  // Outputs: false
System.out.println(x != y);  // Outputs: true
System.out.println(x > y);   // Outputs: false
System.out.println(x < y);   // Outputs: true
System.out.println(x >= y);  // Outputs: false
System.out.println(x <= y);  // Outputs: true

Logical Operators

Logical operators are used to perform logical operations on boolean expressions. Here are the common logical operators:

AND (&&): Returns true if both operands are true.
OR (||): Returns true if at least one of the operands is true.
NOT (!): Returns the opposite of the operand's boolean value.

boolean isSunny = true;
boolean isWarm = false;

System.out.println(isSunny && isWarm);  // Outputs: false
System.out.println(isSunny || isWarm);  // Outputs: true
System.out.println(!isSunny);           // Outputs: false

Conditional Statements (if, else-if, else)

Conditional statements are used to execute different blocks of code based on certain conditions.

int age = 20;

if (age >= 18) {
    System.out.println("You are an adult.");
} else if (age >= 13) {
    System.out.println("You are a teenager.");
} else {
    System.out.println("You are a child.");
}

Ternary Operator

The ternary operator is a shorthand way of writing if-else statements.

int num = 10;
String result = (num % 2 == 0) ? "even" : "odd";
System.out.println(result);  // Outputs: even

Switch Statements

Switch statements allow us to execute different blocks of code based on the value of a variable.

int day = 3;
String dayName;

switch (day) {
    case 1:
        dayName = "Monday";
        break;
    case 2:
        dayName = "Tuesday";
        break;
    // ...
    default:
        dayName = "Invalid day";
}

System.out.println("Today is " + dayName);

Iterative Statements

Iterative statements are used to execute a block of code repeatedly.

For Loops

for (int i = 1; i <= 5; i++) {
    System.out.println("Count: " + i);
}

Enhanced For Each Loop

int[] numbers = {1, 2, 3, 4, 5};

for (int num : numbers) {
    System.out.println(num);
}

While Loops

int i = 1;
while (i <= 5) {
    System.out.println("Count: " + i);
    i++;
}

Do-While Loops

int j = 1;
do {
    System.out.println("Count: " + j);
    j++;
} while (j <= 5);

Exercise

Now after we are done with java fundamentals, lets take all what we've learned so far and package it a nice, simple and basic calculator

import java.util.Scanner;

public class Calculator {
    public static void main(String[] args) {
        Scanner scanner = new Scanner(System.in);

        System.out.println("Welcome to the Simple Calculator!");
        System.out.println("Please enter the first number:");
        double num1 = scanner.nextDouble();

        System.out.println("Please enter the second number:");
        double num2 = scanner.nextDouble();

        System.out.println("Select operation:");
        System.out.println("1. Addition (+)");
        System.out.println("2. Subtraction (-)");
        System.out.println("3. Multiplication (*)");
        System.out.println("4. Division (/)");

        int choice = scanner.nextInt();
        double result = 0;

        switch (choice) {
            case 1:
                result = num1 + num2;
                break;
            case 2:
                result = num1 - num2;
                break;
            case 3:
                result = num1 * num2;
                break;
            case 4:
                if (num2 != 0) {
                    result = num1 / num2;
                } else {
                    System.out.println("Error! Division by zero is not allowed.");
                    return;
                }
                break;
            default:
                System.out.println("Invalid choice!");
                return;
        }

        System.out.println("Result: " + result);

        scanner.close();
    }
}

Section 2 | Object Oriented Programming in java

Java is indeed a versatile programming language that supports multiple programming paradigms, including procedural, functional, and object-oriented programming (OOP). However, the OOP paradigm is the primary one in Java, as it focuses on modeling real-world entities as objects, which have attributes (data) and behaviors (methods).

Let's dive together into the different concepts and aspects of the OOP world in Java.

Absolutely! Let's break down each topic using simple language, analogies, and plenty of code examples.

Understanding Classes and Objects

Think of a class as a blueprint and an object as something built from that blueprint. For example, a class "Car" would define what a car is (its attributes like color, make, and model, and behaviors like driving and honking), while an object would be a specific car built from that blueprint.

Creating and Instantiating Classes

To create a class, you define its attributes and behaviors. To create an object, you use the class blueprint. Here's an example of a class and how to create an object from it:

// Class definition
class Car {
    String color;
    String make;
    String model;

    void drive() {
        System.out.println("The car is driving.");
    }
}

// Creating an object
Car myCar = new Car();

Encapsulation

Encapsulation is like putting your things in a box and only allowing certain people to access them. In Java, you can hide the inner workings of a class and only allow access through specific methods, like getter and setter methods.

class BankAccount {
    private double balance;

    // Getter method
    public double getBalance() {
        return balance;
    }

    // Setter method
    public void setBalance(double amount) {
        balance = amount;
    }
}

Abstraction

Abstraction is like driving a car without needing to know how the engine works. You interact with the car (object) using its interface (methods) without needing to understand its internal implementation.

Constructors

Constructors are like a special recipe used to create an object. They initialize the object's state when it's created. Here's an example:

class Dog {
    String breed;

    // Constructor
    public Dog(String dogBreed) {
        breed = dogBreed;
    }
}

Polymorphism

Polymorphism is like a shape-shifter. An object can take on different forms depending on its context. Method overloading is one example:

class MathOperations {
    // Method overloading
    public int add(int x, int y) {
        return x + y;
    }

    public double add(double x, double y) {
        return x + y;
    }
}

Inheritance

Inheritance is like passing down traits from parent to child. A subclass inherits attributes and behaviors from its superclass. For example:

class Animal {
    void eat() {
        System.out.println("The animal is eating.");
    }
}

class Dog extends Animal {
    void bark() {
        System.out.println("The dog is barking.");
    }
}

Static Members

Static members are like shared resources among all objects of a class. They belong to the class itself rather than any specific object. For example:

class Circle {
    static final double PI = 3.14;

    static double calculateArea(double radius) {
        return PI * radius * radius;
    }
}

Absolutely! Let's delve into each aspect of inheritance in Java with detailed explanations, code examples, and analogies.

Overview of Inheritance

Inheritance is like passing down traits from parents to their children. In Java, it allows a class (subclass) to inherit attributes and behaviors from another class (superclass). This promotes code reuse and helps in organizing and managing related classes.

The Object Class and Its Core Methods

In Java, every class implicitly extends the Object class. The Object class provides several core methods that are available to all classes. Some of these methods include:

toString(): Returns a string representation of the object.
equals(Object obj): Indicates whether some other object is "equal to" this one.
hashCode(): Returns a hash code value for the object.

class MyClass {
    int value;

    // toString method override
    @Override
    public String toString() {
        return "Value: " + value;
    }
}

MyClass obj = new MyClass();
obj.value = 10;
System.out.println(obj.toString());  // Outputs: Value: 10

Object Comparison

Java provides the equals() method for comparing objects for equality. By default, it compares memory references. However, you can override this method in your classes to provide custom comparison logic.

class Person {
    String name;
    int age;

    @Override
    public boolean equals(Object obj) {
        if (this == obj)
            return true;
        if (obj == null || getClass() != obj.getClass())
            return false;
        Person person = (Person) obj;
        return age == person.age && Objects.equals(name, person.name);
    }
}

Method Overriding

Method overriding allows a subclass to provide a specific implementation of a method that is already defined in its superclass. This is useful for providing specialized behavior.

class Animal {
    void makeSound() {
        System.out.println("Animal makes a sound");
    }
}

class Dog extends Animal {
    @Override
    void makeSound() {
        System.out.println("Dog barks");
    }
}

Abstract Classes

Abstract classes are like blueprints for other classes. They cannot be instantiated themselves but can have abstract methods that must be implemented by subclasses.

abstract class Shape {
    abstract double area();
}

class Circle extends Shape {
    double radius;

    @Override
    double area() {
        return Math.PI * radius * radius;
    }
}

Final Classes

Final classes are like finished products that cannot be modified or extended. You can't create subclasses of a final class.

final class FinalClass {
    // Class definition
}

// Error: Cannot inherit from final class
class SubClass extends FinalClass {
}

Interfaces in Java

Think of an interface as a contract or a set of rules that a class must follow. It defines a list of methods that implementing classes must provide, but it doesn't contain any implementation itself.

// Interface definition
interface Animal {
    void makeSound();
    void eat();
}

// Implementing class
class Dog implements Animal {
    @Override
    public void makeSound() {
        System.out.println("Dog barks");
    }

    @Override
    public void eat() {
        System.out.println("Dog eats bones");
    }
}

In this example, the Animal interface defines two methods: makeSound() and eat(). The Dog class implements these methods according to the contract specified by the Animal interface.

Key Features and Usage Scenarios

Interfaces provide several key features and are commonly used in various scenarios:

Multiple Inheritance: Unlike classes, a Java class can implement multiple interfaces. This allows a class to inherit behavior from multiple sources.

interface Flyable {
    void fly();
}

class Bird implements Animal, Flyable {
    // Implement methods from both Animal and Flyable interfaces
}

Polymorphism: Interfaces enable polymorphic behavior, allowing objects to be treated as instances of their interface type. This promotes flexibility and code reusability.

void performAction(Animal animal) {
    animal.makeSound();
    animal.eat();
}

Loose Coupling: Interfaces facilitate loose coupling between components of a system. Classes interact with each other through interfaces rather than concrete implementations, making the system more modular and easier to maintain.

interface DataAccess {
    void saveData(String data);
}

class DatabaseAccess implements DataAccess {
    @Override
    public void saveData(String data) {
        // Save data to the database
    }
}

class FileAccess implements DataAccess {
    @Override
    public void saveData(String data) {
        // Save data to a file
    }
}

In this example, different classes can implement the DataAccess interface to provide different ways of saving data, such as to a database or a file.

Interfaces serve as powerful tools in Java programming, providing a way to define contracts, enable polymorphic behavior, and promote loose coupling between components. They are essential for building modular, flexible, and maintainable software systems.

Exercise

using all what we learned in this section about OOP in java let's rebuild our calculator in object oriented style:

// Interface for basic calculator operations
interface Calculator {
    double calculate(double num1, double num2);
}

// Addition operation
class Addition implements Calculator {
    @Override
    public double calculate(double num1, double num2) {
        return num1 + num2;
    }
}

// Subtraction operation
class Subtraction implements Calculator {
    @Override
    public double calculate(double num1, double num2) {
        return num1 - num2;
    }
}

// Multiplication operation
class Multiplication implements Calculator {
    @Override
    public double calculate(double num1, double num2) {
        return num1 * num2;
    }
}

// Division operation
class Division implements Calculator {
    @Override
    public double calculate(double num1, double num2) {
        if (num2 != 0) {
            return num1 / num2;
        } else {
            throw new IllegalArgumentException("Error! Division by zero is not allowed.");
        }
    }
}

// Calculator class encapsulating calculator operations
class BasicCalculator {
    private Calculator calculator;

    public BasicCalculator(Calculator calculator) {
        this.calculator = calculator;
    }

    public double performCalculation(double num1, double num2) {
        return calculator.calculate(num1, num2);
    }
}

public class CalculatorApp {
    public static void main(String[] args) {
        BasicCalculator calculator = new BasicCalculator(new Addition());
        double result = calculator.performCalculation(10, 5);
        System.out.println("Addition Result: " + result);

        calculator = new BasicCalculator(new Subtraction());
        result = calculator.performCalculation(10, 5);
        System.out.println("Subtraction Result: " + result);

        calculator = new BasicCalculator(new Multiplication());
        result = calculator.performCalculation(10, 5);
        System.out.println("Multiplication Result: " + result);

        calculator = new BasicCalculator(new Division());
        result = calculator.performCalculation(10, 5);
        System.out.println("Division Result: " + result);
    }
}

Section 3 | Advance Java Topics

Exception Handling?

Think of exceptions as unexpected events that can happen while your Java program is running. Just like when you're playing a game and suddenly something unexpected happens, like the power going out or your controller running out of batteries.

Checked Exceptions:

These are like warning signs that you can anticipate. It's like knowing that if you're playing with a ball indoors, it might break something. So, you handle it by playing more carefully. In Java, you're forced to handle these kinds of exceptions, either by catching them or declaring that your method might throw them.

Unchecked Exceptions:

These are the sneaky surprises you didn't see coming. It's like accidentally stepping on a Lego piece while walking barefoot. Ouch! These exceptions are not forced to be handled explicitly, but you should still try to handle them to keep your program safe.

Try-Catch Blocks:

Imagine you're catching a ball. You extend your hands, try to catch it, and if you miss, it falls on the ground. In Java, you use try to attempt something that might throw an exception, and then you catch the exception if it happens, just like catching a ball to prevent it from crashing on the ground.

try {
    // Code that might throw an exception
} catch (ExceptionType e) {
    // Code to handle the exception
}

Finally Block:

This is like a safety net. No matter what happens, this block of code always runs. It's like cleaning up after playing with your toys. In Java, you use finally to ensure that some code gets executed, whether an exception occurs or not.

try {
    // Code that might throw an exception
} catch (ExceptionType e) {
    // Code to handle the exception
} finally {
    // Code that always runs
}

Throw and Throws:

Sometimes you want to be the one causing the exception. It's like saying, "I'm not playing anymore, and I'm throwing the ball back." In Java, you use throw to throw an exception manually. And throws is like passing the responsibility to someone else. You declare that your method might throw a certain type of exception, leaving it to the caller to handle.

void myMethod() throws MyException {
    // Code that might throw MyException
}

// Somewhere else in the code
try {
    myMethod();
} catch (MyException e) {
    // Handle MyException
}

Generics:

Generics in Java are like magical containers that can hold any type of data. They provide flexibility and type-safety to your code, ensuring that you can work with different types seamlessly. Let's dive deeper into the magical world of generics!

Generics in Classes:

Imagine a treasure chest that can hold various treasures – gold coins, precious gems, or ancient artifacts. In Java, you can create generic classes using <T>, where T represents a type parameter.

public class TreasureChest<T> {
    private T treasure;

    public void store(T item) {
        this.treasure = item;
    }

    public T retrieve() {
        return this.treasure;
    }
}

Generics in Methods:

Now, envision a wand that can cast spells on any object. In Java, you can create generic methods that operate on different types of data, enhancing code reusability and flexibility.

public class Magic {
    public <T> void castSpell(T target) {
        // Perform magic on the target
    }
}

Wildcards:

Wildcards in Java generics are like special lenses that allow you to view your code from different perspectives. Let's venture deeper into the jungle of generics and uncover the secrets of wildcards!

Upper Bounded Wildcards (`<? extends T>`):

Imagine a magical net that captures all creatures with specific characteristics. In Java, <? extends T> captures all objects of a type that is a subtype of T, allowing you to work with a broad range of data.

public void feedHerbivores(List<? extends Animal> herbivores) {
    // Feed the herbivores
}

Lower Bounded Wildcards (`<? super T>`):

Now, picture a protective barrier that shields against specific dangers. In Java, <? super T> specifies a lower bound for the wildcard and captures all objects that are superclasses of T, ensuring safety and security in your code.

public void protectAgainstPredators(List<? super Lion> protections) {
    // Set up protections against predators
}

Generic Methods:

Just as a skilled magician can perform various tricks with a single wand, generic methods in Java can operate on different types of data, enhancing code flexibility and versatility.

public <T> void performAction(T item) {
    // Perform action on the item
}

Generic Classes with Multiple Type Parameters:

Imagine a treasure map with multiple clues leading to hidden riches. In Java, you can create generic classes with multiple type parameters, allowing you to handle various types of data simultaneously.

public class TreasureMap<X, Y> {
    private X clue1;
    private Y clue2;

    // Constructor, getters, and setters
}

Collections:

Imagine you're on a quest to collect rare treasures scattered across a magical kingdom. Java collections are like your trusty backpacks, equipped with special compartments to store and organize your treasures. Whether it's shiny gems, ancient artifacts, or mystical potions, collections in Java provide a versatile way to manage and manipulate your data.

Collections are just Interfaces!:

In Java, collections are represented by a variety of interfaces, each with its unique characteristics and abilities. Think of interfaces as blueprints for different types of backpacks, each designed for specific purposes.

List<String> backpack = new ArrayList<>();  // A backpack for storing a list of treasures
Set<Integer> pouch = new HashSet<>();       // A pouch for storing unique treasures
Map<String, Integer> chest = new HashMap<>();// A chest for storing treasures with keys

Collection Types:

Lists:

Imagine a list of treasures neatly arranged in a row, like books on a shelf. In Java, lists allow you to store a collection of elements in a specific order, making it easy to access and manipulate them.

List<String> backpack = new ArrayList<>();
backpack.add("Golden Sword");
backpack.add("Enchanted Ring");
backpack.add("Magic Potion");

Sets:

Now picture a set of unique treasures, like a collection of rare gems. In Java, sets ensure that each element is unique, preventing duplicates and providing fast retrieval.

Set<String> pouch = new HashSet<>();
pouch.add("Ruby");
pouch.add("Emerald");
pouch.add("Sapphire");

Maps:

Finally, envision a treasure map with clues leading to hidden riches. In Java, maps associate keys with values, allowing you to retrieve treasures based on their unique identifiers.

Map<String, Integer> chest = new HashMap<>();
chest.put("Gold Coins", 100);
chest.put("Silver Bars", 50);
chest.put("Diamonds", 10);

Iterating Through Collections:

Just as you explore the vast landscapes of a kingdom, you can traverse through collections using iterators, uncovering treasures along the way.

for (String treasure : backpack) {
    System.out.println("Found: " + treasure);
}

Sorting Collections:

Imagine arranging your treasures in order of value, from least to most precious. In Java, you can sort collections using comparators, ensuring that your treasures are organized efficiently.

Collections.sort(backpack);

Manipulating Collections:

Now envision combining two collections to create an even larger collection of treasures. In Java, you can manipulate collections using various methods, such as merging, filtering, and transforming them.

List<String> additionalTreasures = new ArrayList<>();
additionalTreasures.add("Magic Wand");
backpack.addAll(additionalTreasures);

Functional Programming:

Imagine you're a wizard in a magical kingdom, wielding powerful spells to manipulate the forces of nature. Functional programming in Java is like mastering the ancient art of spellcasting, allowing you to perform powerful transformations on your data with ease.

Lambda Expressions:

In Java, lambda expressions are like enchanted spells, allowing you to encapsulate behavior and pass functions as parameters to other methods.

// Traditional way of defining a Runnable
Runnable runnable = new Runnable() {
    @Override
    public void run() {
        System.out.println("Hello, world!");
    }
};

// Using lambda expression
Runnable runnable = () -> {
    System.out.println("Hello, world!");
};

Suppliers:

Picture a mystical shopkeeper who can conjure any item you desire out of thin air. In Java, suppliers are like that shopkeeper, providing a way to lazily generate or supply values when needed.

Supplier<Integer> randomNumber = () -> (int) (Math.random() * 100);
System.out.println(randomNumber.get());  // Output: A random number between 0 and 100

Predicates:

Imagine you're a wise wizard who can discern truth from falsehood with a single glance. In Java, predicates are like your magical eyes, allowing you to test conditions and filter elements based on those conditions.

Predicate<Integer> isEven = num -> num % 2 == 0;
System.out.println(isEven.test(4));  // Output: true
System.out.println(isEven.test(5));  // Output: false

Consumers:

Now, envision a benevolent fairy who consumes your worries and leaves behind a sense of peace. In Java, consumers are like that fairy, accepting values and performing actions on them without returning anything.

Consumer<String> printUpperCase = str -> System.out.println(str.toUpperCase());
printUpperCase.accept("magic");  // Output: MAGIC

Functions:

Finally, imagine a magical alchemist who transforms one substance into another. In Java, functions are like that alchemist, taking one input and producing another output based on some transformation.

Function<Integer, String> convertToString = num -> "Number: " + num;
System.out.println(convertToString.apply(42));  // Output: Number: 42

Java Streams:

In Java, streams are sequences of elements that support various methods to perform operations on those elements. Think of them as magical conduits that flow through your data, allowing you to perform actions on each element as it passes through.

Functional Interfaces in Streams

Streams use functional interfaces to perform their work. These interfaces, such as consumers, predicates, suppliers, and functions, provide blueprints for behaviors that can be passed to stream operations.

Consumers in streams:

Consumers are like magical observers that accept a single input and perform some action on it without returning anything. In streams, consumers are used to perform terminal operations that consume the elements of the stream.

List<String> ingredients = Arrays.asList("Eye of newt", "Wing of bat", "Tooth of wolf");

// Printing each ingredient using a consumer
ingredients.stream()
           .forEach(System.out::println);

Predicates in streams:

Predicates are like magical filters that test conditions and return true or false. In streams, predicates are used to filter elements based on certain criteria.

// Filtering ingredients that contain "bat"
ingredients.stream()
           .filter(ingredient -> ingredient.contains("bat"))
           .forEach(System.out::println);

Suppliers in streams:

Suppliers are like magical providers that lazily generate or supply values when needed. In streams, suppliers are not as commonly used directly, but they play a role in certain stream operations.

// Generating a stream of random numbers using a supplier
Stream<Integer> randomNumberStream = Stream.generate(() -> (int) (Math.random() * 100));

Functions in streams:

Functions are like magical transformers that take one input and produce another output based on some transformation. In streams, functions are used for mapping elements to another form.

// Converting ingredients to uppercase using a function
ingredients.stream()
           .map(String::toUpperCase)
           .forEach(System.out::println);

Concurrency and Multithreading:

Concurrency is the ability of a program to execute multiple tasks simultaneously, allowing it to make progress on more than one task at a time.

What are Threads?

Threads are like individual sailors on your ship, each responsible for carrying out a specific task. In Java, you can create and manage threads to perform tasks concurrently.

Creating Threads:

In Java, you can create threads by extending the Thread class or implementing the Runnable interface. Let's create a simple thread that prints a message.

class MyThread extends Thread {
    public void run() {
        System.out.println("Hello from MyThread!");
    }
}

public class Main {
    public static void main(String[] args) {
        MyThread thread = new MyThread();
        thread.start(); // Start the thread
    }
}

Implementing Runnable:

Alternatively, you can implement the Runnable interface and pass it to a Thread object.

class MyRunnable implements Runnable {
    public void run() {
        System.out.println("Hello from MyRunnable!");
    }
}

public class Main {
    public static void main(String[] args) {
        Thread thread = new Thread(new MyRunnable());
        thread.start(); // Start the thread
    }
}

Synchronization:

Imagine your sailors trying to access a shared resource, like a treasure chest, simultaneously. Synchronization in Java ensures that only one thread can access a critical section of code at a time, preventing conflicts and ensuring thread safety.

class Counter {
    private int count = 0;

    public synchronized void increment() {
        count++;
    }

    public synchronized int getCount() {
        return count;
    }
}

Executors:

Think of executors as managers who oversee a group of sailors (threads) and assign them tasks efficiently. Executors provide a higher-level abstraction for managing threads and executing tasks asynchronously.

ExecutorService executor = Executors.newFixedThreadPool(5);
executor.submit(new MyTask());
executor.shutdown();

Conclusion

In this comprehensive guide, we've journeyed through the fundamental concepts of Java programming, explored the intricacies of object-oriented programming (OOP), and delved into advanced topics that unlock the full potential of Java. From setting up your development environment to mastering control flow, classes, objects, inheritance, interfaces, and beyond, you've gained a solid understanding of Java's core principles and features.

As we conclude our exploration of Java 101, it's clear that you've taken significant strides on your path to becoming a proficient Java programmer. Armed with this knowledge, you're well-equipped to tackle real-world coding challenges and embark on exciting projects with confidence.

But the journey doesn't end here. In our next adventure, we'll delve into the fascinating world of data structures and algorithms—the cornerstone of computer science and software development. Together, we'll unravel the mysteries of data organization, algorithmic efficiency, and problem-solving techniques that will elevate your coding skills to new heights.

So, stay tuned for our next chapter as we continue our quest for mastery in the realms of programming and software development. Until then, keep coding, keep learning, and let your passion for Java ignite your journey towards excellence!

Does Data Structures and Algorithms Matter in Backend? No BS Answer!

Louai Boumediene — Fri, 19 Apr 2024 17:54:06 +0000

Introduction:

Have you ever wondered whether data structures and algorithms (DSA) truly matter in backend engineering? The answers you've heard might seem vague or inconclusive and they all say "It depends...", "It's needed for interview never needed later...". But fear not, because I'm here to provide a clear answer.

DSA is a hot topic in the tech world, but its relevance can sometimes feel elusive. In this blog, we're going to unravel the mystery and discover the real importance of DSA in backend development.

Integrator vs Core Backend Engineer

In my humble opinion that is mainly inspired form a YouTube video titled: How Important are Algorithms and Data Structures in Backend Engineering? by Hussein Nasser.

I deeply believe that in the realm of backend engineering, there is two types of engineers

Integrator: Who resembles 80% of backend engineers out there, and most of backend engineering roles in the market are integrating roles! what I mean by integrator backend engineer is the person who brings together various backend components to create functional applications. They integrate web frameworks with databases, set up APIs, and configure servers to deliver seamless user experiences. While they will not delve as deeply as core backend engineers into the intricacies of how different components they are integrating, like as a backend engineer who is responsible for developing the backend infrastructure for a SaaS product he is not obliged to understand how Database engines implement B-tree or how the web framework he is using is built form the ground-up.
Core Backend Engineer: And those represent the rest 20% of backend engineers, who are responsible for building the core components that integrators combine together, like the engineers building and maintaining MongoDB database engine or the engineers behind Nginx web server and examples are endless.

When you really need to master DSA:

short answer is When you're responsible for building foundational components like database engines, a deep understanding of DSA becomes indispensable.
For instance, if you're developing a database engine, you must grasp concepts like B-Trees and hashing. These tools enable efficient data organization and retrieval, ensuring the smooth operation of the database.
On the other hand, if your role involves integrating existing databases into applications, the need for mastering DSA diminishes. While a basic understanding might be helpful for optimizing queries or managing data efficiently, it's not a prerequisite for success.

Embarking on my DSA Learning Journey:
Data Structures and Algorithms is a huge topic that people are getting PHDs doing research on! That's why I charted out a basic path to (DSA) learning, inspired by resources like:

and my path will be as follows:

Picking a Programming Language: The beauty of learning DSA is that you can pick almost any programming language to get started. Whether it's Python, Java, C#, C, or Rust, the principles remain the same. For me, Java holds a special place—it's a language I'm comfortable with and enjoy coding in.
Understanding Algorithm Complexity: Before diving into specific data structures and algorithms, it's crucial to grasp the fundamentals. Understanding algorithm complexity and it's measuring systems like "Big O Notation" helps you evaluate the efficiency of your code and make informed decisions.
Basic Data Structures: Start with the basics—the building blocks of DSA. Arrays, linked lists, stacks, queues, and hash tables form the foundation of many algorithms and data structures.
Sorting Algorithms: Sorting is a fundamental operation in computer science, and there are various algorithms to achieve it. Learning sorting algorithms like bubble sort, insertion sort, and quicksort equips you with valuable problem-solving skills.
Search Algorithms: Searching for elements within data structures is another essential skill. Dive into search algorithms like linear search, binary search, and interpolation search to understand their workings and applications.
Tree Data Structures: Trees are hierarchical structures that find applications in various domains. Explore binary trees, binary search trees, B-trees, AVL trees, and related algorithms to understand their nuances and optimize operations.
Graph Data Structures: Graphs are versatile structures used to model relationships between objects. Understand directed and undirected graphs, along with algorithms like depth-first search (DFS) and breadth-first search (BFS), to tackle graph-related problems effectively.

Simple DSA Roadmap I made

Consusion:

DSA isn't just a subject—it's a way of thinking. That's why I'm on a journey to learn it continuously, alongside other backend topics and I'll make it kind of sub-journey of hte main backend journey. Each milestone I reach in DSA will be a stepping stone, documented in a new blog post. Next Blog will be Java 101.

Unveiling the Backbone: Dive Deep into Backend Engineering Essentials!

Louai Boumediene — Thu, 11 Apr 2024 15:23:58 +0000

Introduction

Even after diving into building basic backend CRUD apps and moving beyond the initial thrill, I still find myself baffled by terms like "Distributed systems." Despite my efforts with Googling and tutorials, concepts like the importance of tools such as Kafka or RabbitMQ, understanding Microservices, and why gRPC is prevalent in this architecture continue to elude me. There's a lot to unpack, and it leaves me scratching my head.

# complete BS!
backend_development == the_framework

My previous idea about backend engineering

I don't blame myself, or the many other new developers navigating the backend landscape, for our shallow understanding of the subject. The reality is, if you search for "Learn Backend Development/Engineering" on major learning platforms like YouTube, Coursera, or Udemy, you'll likely find that 90% of the results are titled something like:

"Learn Backend Development with Django"
"Master Node.js Backend Development"
"The Spring Boot Backend Development Full Course"

It's no wonder many of us struggle to grasp the broader concepts when the focus is often on specific frameworks or technologies. But isn't backend development more than just mastering a programming language or framework? Don't get me wrong though, there is some awesome people in the community who deeply explore the subject. Later on, I'll mention some of them who have really helped me gain a better understanding of things around me thus far.

My new understanding of backend engineering

In my exploration of backend engineering, I'm delving into crucial components that drive complex backend infrastructure. We'll delve into topics such as how data flows between clients and servers, storing information in databases securely, and maintaining overall system safety. Additionally, we'll explore tools designed to enhance performance and streamline development processes.
The following list of essential backend concepts that i will be exploring, learning, and implementing in my journey:

Communication Protocols:
Facilitate data exchange between client and server (e.g., HTTP for web browsing, WebSocket for real-time communication).
Web Servers:
Handle client requests, serve web content, and execute server-side logic to generate dynamic web pages.
Databases:
Store and retrieve data, with relational databases using structured tables and non-relational databases offering flexible data models.
Proxies:
Act as intermediaries between clients and servers, improving security, performance, and scalability by playing various roles from caching to load balancing to others.
Caching:
Enhance performance and reduce server load by storing frequently accessed data in temporary storage for quick retrieval.
Message Queues:
Enable asynchronous communication between distributed components, allowing systems to process tasks independently and efficiently.
Web Frameworks:
Simplify backend development by providing pre-built components and structures to streamline the creation of web applications.
Security:
Ensure the protection of backend systems from threats through encryption, firewalls, and authentication mechanisms and security protocols.

Foundations matter

Prioritize Concepts Over Frameworks and Languages, they are just tools at the end!

One of the most valuable lessons I've learned so far is to never rely on any specific framework or programming language. Instead, I aim to be what I like to call a "Framework/Language-agnostic engineer." This means having a solid foundation in the underlying and universal concepts of backend or computer science in general, which can be easily applied in any language or framework.

For instance, understanding concepts like concurrency and asynchronous I/O in pure computer science terms before diving into specific implementations like Python's "asyncio" library or JavaScript promises is crucial. Similarly, grasping the idea of web APIs, whether they're RESTful, gRPC, or GraphQL, and then learning an implementation like Express or Django Rest Framework is essential.

Too often, aspiring backend developers take the reverse path, confining themselves to a specific technology and getting locked into a shallow, syntax-level understanding of the subject. This can leave them struggling to build complex apps or debug simple ones when serious problems arise. Even worse, they may become entrenched in one implementation or design philosophy in an ever-changing field like backend engineering, where yesterday's new technologies are today's old ones. The only constant is the concept behind them.

This principle has contributed significantly to where I am today. A few months ago, I applied for a frontend position at the company I work for now. At the time, they didn't really need frontend developers. However, I ended up getting a backend developer position despite having very limited experience in backend and not even with the tech stack they use. This was mainly because I showed the interviewer that for me:

Technologies are just tools that come and go, and what really matters is the ideas and concepts behind them.

Resources That helped me a lot

Roadmap.sh: This website is like a treasure map for backend development! It lays out a clear path with lots of helpful resources to guide you along the way.
Hussein Nasser: Oh, this guy is awesome! Hussein Nasser talks about all things backend in a way that's easy to understand. He's not just about coding; he dives deep into why things work the way they do and gives real-life examples.
[Hussein Nasser YouTube Channel]
Amigos Code: If you're into Java and Spring framework, you'll love Amigos Code. They break down backend development into simple steps and give practical advice without any fancy promises. It's like chatting with a friend who knows a lot about coding!

Conclusion

As we wrap up our exploration of key backend engineering concepts, it's clear that there's still much more to learn. From communication protocols to security measures, each concept we've discussed offers a glimpse into the complexity of backend development.

In our future blogs, we'll dive deeper into these topics, exploring them in greater detail and building practical projects to deepen our understanding. Join me on this journey as we continue to uncover the secrets of backend engineering and work towards building robust, efficient, and secure web applications together. Stay tuned for more exciting discoveries ahead!

Embarking on my Back-end Journey: The Intent Behind this Blog

Louai Boumediene — Sat, 06 Apr 2024 20:36:19 +0000

Introduction

Let me take you back to when the world seemed upside down during the late 2020s, with the coronavirus causing chaos everywhere. At 17, I dipped my toes into the world of coding. I started with the very basics of Python, making simple apps and games that brought me joy, even though I didn't fully understand what I was doing.

As time went on, I explored different parts of the tech world. From making apps for Android using Java, to trying my hand at creating games using C# and Unity 3D, to learning the very basics of ethical hacking and cyber security.

I bounced around a lot, but never really understood anything in depth. Until something clicked when I discovered web development. The idea of how everything is connected online fascinated me. So, I committed to focusing on web development and started learning the basics of the web: HTML, CSS, and JavaScript.

Gone were the days of blindly copying tutorials. I began to understand the stuff I was working with. By the time I turned 19, I'd learned a lot. I got better at planning my studies, finding resources online, and fixing problems I ran into. I had some level of proficiency in web development that I was able to build simple full-stack web app using React, Express, and MongoDB. I built a solid portfolio as a junior developer with a couple of acceptable projects that proved my solid understanding of how different web components are connected together (Front-end + Back-end + Database). Not too much though, but enough to land my first job as a junior back-end developer in January 2024 at age of 20, nearly three years since I wrote my first line of code.

I ran through a lot more than the mentioned above, like having at some point of time two published apps on the Play Store, with more than 250,000+ downloads on both of them, but sad enough I got my Google Play Console account terminated due to a few stupid mistakes I did. The latter was too hard, but I made my way through it after some time, and eventually, it was what led to a huge shift both mentally and career-wise, from Android to web development.

The most significant lesson gleaned from my journey thus far is:

"Learning a bit of everything may kickstart your journey and provide a broad overview of the various puzzle pieces around you, but it will never propel you far."

After commencing my current role as a back-end developer, surrounded by intelligent, talented, and humble colleagues, this lesson evolved into:

"Strive to understand a bit of everything and everything about one thing."

And that one thing I am committed to mastering is back-end engineering.

Why I'm starting this blog series

One of the most frequent pieces of advice I've heard from countless people in the community is to "learn in public," and that's precisely what I aim to do from now on. While I'm not entirely new to backend development, I've come to realize that I've never truly understood what backend development entails. This realization has sparked a curiosity within me, prompting me to delve deeper into the world of backend engineering.

This blog series isn't intended to be educational in the traditional sense. Instead, it's a documentation of my personal journey in back-end engineering over the next six months.

The Path Ahead

Over the next six months, I'll be embarking on a weekly blogging journey, sharing insights and discoveries from my ongoing exploration of back-end engineering. Each blog post will offer a glimpse into my journey, focusing on what I've learned, researched, and worked on during the previous week. As I navigate through the vast landscape of back-end technologies, my blog posts will primarily fall into the following categories:

Lessons and Takeaways

From time to time, I'll delve into non-technical topics, sharing valuable lessons and takeaways gleaned from my experiences along the way. These reflections will offer insights into the broader aspects of personal and professional growth.

Learning Core Back-end Technologies

My learning journey will revolve around mastering core back-end technologies, one step at a time. Whether it's communication protocols, databases, messaging systems, or data structures and algorithms, I'll dive deep into each topic, unraveling its complexities and uncovering its practical applications.

Learning & Research

In this phase, I'll immerse myself in the fundamentals of each topic, striving to grasp the underlying theories and principles. For example, when exploring databases, I won't dive straight into specific systems like Postgres or MongoDB. Instead, I'll lay the groundwork by delving into the broader concepts of database management, understanding their significance and functionality.

Learning Applications of the Topic

Building upon my theoretical understanding, I'll then shift gears to explore real-world applications of the topic at hand. Using databases as an example, I'll delve into the intricacies of systems like Postgres and MongoDB, examining their features, use cases, and best practices.

Implementation

Finally, armed with newfound knowledge, I'll put theory into practice by embarking on hands-on implementation projects. For instance, following our exploration of databases, I might undertake the challenge of building a CRUD application using Postgres or MongoDB. Alternatively, I might push the boundaries further by creating a simplified version of a database system from scratch, utilizing my understanding of low-level implementation and algorithms.

Through this structured approach, I aim to not only deepen my understanding of back-end technologies but also to share my insights and experiences with fellow learners and enthusiasts. Together, let's embark on this exciting journey of discovery and growth in the realm of back-end engineering.

Conclusion

As I finish up telling you about my journey into back-end stuff, I'm pretty excited about what's next. From being a total beginner to starting to figure out back-end stuff, I've learned a lot.

In this blog series, I'm gonna share what I learn, the problems I face, and the cool stuff I discover as I dive into backend engineering. We'll talk about important stuff and how it works in real life.

So, come along with me on this adventure as we explore backend stuff together. We'll learn, grow, and have fun, all while figuring out this tech world.

Here's to the journey ahead, full of new things to learn and explore. Let's keep going together!