DEV Community: shema

When and Why You Need Sharding: A Complete Guide to Scaling Databases Efficiently

shema — Sun, 12 Jan 2025 08:51:42 +0000

Imagine your website or app is growing rapidly, and suddenly, it’s not just a few hundred users anymore — it's thousands, maybe millions. Your database which was once fast and efficient, starts to slow down trying to handle many requests. The data starts piling up in one giant pile, making it harder and slower to access. You can almost hear your database struggling to keep up. This is where Sharding comes in to save the day and to the rescue.

What is Sharding now?

Sharding is like organizing a massive library. Instead of putting every book in one gigantic pile that takes forever to search through, you divide the books into different sections and each section is called a shard. Each section has its own shelf, making it easier to find what you're looking for faster.

In technical terms, sharding splits your database into smaller, more manageable parts, called shards, and stores each shard on a different server. This allows servers to handle distinct pieces of data, speeding up access and preventing your system from being overwhelmed. It's a strategy used while dealing with large volumes of data or a growing number of users.

How Does Sharding Work?

Think of sharding as dividing a giant Excel sheet into smaller sheets. You can split the data in various ways for instance:

• Range-Based Sharding: Here you divide your data into chunks based on a specific range may be alphabetically or numerically. For example, all data from A-M (See on the above figure) might go to one server, and N-Z (See on the above figure) might go to another.

• Hash-Based Sharding: A hash function assigns data to shards based on a key like user ID. This ensures a uniform distribution of data across servers.

• Directory-Based Sharding: A central directory keeps track of where each piece of data is stored, helping the system know where to look for the information.

When do you need Sharding?

Sharding isn’t necessary for everyone, but here’s when you’ll definitely need it:

Massive Amounts of Data: If your database grows too large to fit on a single server, sharding becomes essential. As your data volume increases, a single server will struggle to store, retrieve, and manage it all efficiently.
High Traffic or User Load: When you have thousands or millions of users making requests simultaneously, a single server cannot resist. Sharding allows you to distribute the load across multiple servers, so each one handles fewer requests, leading to faster response times.
Performance Bottlenecks: As your data grows, certain queries might start taking longer to execute. Sharding can reduce the time it takes to retrieve information because each shard only contains a smaller subset of the data. This makes querying faster and more efficient.
Scaling Your Business: If you’re rapidly scaling your product or service, sharding gives you the flexibility to add more servers and distribute data across them, ensuring that the system can grow without breaking.

Sharding is a powerful tool that helps you scale your system to handle more data, traffic, and users. When your database starts to grow or slow down, sharding can be the key to improving performance and keeping everything running smoothly. By splitting the workload across multiple servers, you ensure that your system stays efficient and ready for future heavy loads.

Thanks for reading my post and taking this far. Hopefully you've learnt one thing or another. Please, your comment or reaction on this post is incredible.

Uncovering mystery in software development

shema — Thu, 02 Jan 2025 08:18:31 +0000

Application Logic vs. Business Logic: Key Differences with Simple Examples

shema ・ Jan 2

#javascript #webdev #tutorial #programming

Application Logic vs. Business Logic: Key Differences with Simple Examples

shema — Thu, 02 Jan 2025 07:36:54 +0000

Introduction

Have you ever used an app that just works, calculates prices perfectly, follows complex rules seamlessly, and delivers exactly what you need? Now imagine building such an app. Where do you start? What ensures that every calculation is spot-on while the system runs like clockwork? The answer lies in mastering two essential pillars of software development: business logic and application logic.

These two concepts are the invisible forces behind the apps and websites we use every day. But here’s the thing: mixing them up can turn your clean, efficient codebase into an unmanageable nightmare. It is the source of evil too in software development land. Knowing the difference isn’t just a nice-to-have skill, it’s a game-changer for creating scalable, maintainable software.

In this article, we’ll take you on a journey to demystify application logic and business logic. With simple and easy to follow, real-world examples and clean code snippets, you’ll quickly understand their roles and see why separating them is the secret to software success. Whether you’re a beginner exploring the basics or a developer refining your craft, this guide will give you clarity and confidence.

Ready to transform how you think about software design? Let’s dive in and unlock the power of clean, logical code!

In software development,business logic and application logic play distinct roles. Business logic defines the rules and operations that make your software valuable by solving real-world problems. For instance, calculating a customer discount is a business rule. On the other hand, application logic manages the interaction and workflow of your system, connecting various components like databases, APIs, and user interfaces to execute the business rules.

What is Business Logic?

Business logic is the core of your application that handles what needs to happen based on rules. For example, in an e-commerce platform, applying a 10% discount for orders above $100 is a business rule.

Below is a simple JavaScript function to demonstrate:

// Business Logic: Apply a 10% discount if total is greater than $100
function calculateDiscount(totalAmount) {
  return totalAmount > 100 ? totalAmount * 0.1 : 0; // 10% discount
}

// Business Logic: Calculate the final amount after discount
function calculateFinalAmount(totalAmount) {
  const discount = calculateDiscount(totalAmount);
  return totalAmount - discount;
}

These functions focus only on the rules and calculations, without worrying about where the data comes from or how it will be displayed.

What is Application Logic?

Application logic deals with how things work, ensuring the business logic is applied in the right context. It handles fetching data, calling the appropriate business logic functions, and managing the workflow. For example, retrieving a user’s cart total, calculating the discount, and showing the results.

Here's how it looks:

// Application Logic: Fetching data, using business logic, and showing results
function processOrder(userId) {
  const cartTotal = fetchCartTotal(userId); // Simulate fetching cart total
  const discount = calculateDiscount(cartTotal); // Apply business logic
  const finalAmount = calculateFinalAmount(cartTotal); // Calculate final amount

  console.log(`User ID: ${userId}`);
  console.log(`Cart Total: $${cartTotal}`);
  console.log(`Discount: $${discount}`);
  console.log(`Final Amount to Pay: $${finalAmount}`);
}

// Mock function to simulate fetching a user's cart total
function fetchCartTotal(userId) {
  return 120; // Ex: User's cart total is $120
}

// Function invocation(calling) for execution.
processOrder(1);

This function connects different parts of the application: fetching data, applying business logic, and displaying results.

Why Separate Them?

Separating business logic and application logic makes your code maintainable, reusable, and testable. Business rules can be reused across multiple areas of the application (ex: APIs, admin dashboards) without modification. If the discount rule changes, you only need to update the business logic. Similarly, application logic ensures smooth integration and flow, independent of rule changes.

By maintaining this separation, developers can build scalable and modular systems that are easier to debug and enhance over time.

Understanding Middleware: The Essential Guide For Developers

shema — Thu, 26 Dec 2024 17:12:10 +0000

Do we need Middleware? why?

Middleware acts as a bridge between the user and your app, handling essential tasks like authentication, logging, and data formatting. It ensures requests are properly prepared before reaching the core of your application.

What Is Middleware?

Think of middleware as a receptionist in an office. Before you meet with the manager (your app), the receptionist (middleware) checks your ID, verifies permissions, and ensures you're in the right room. This allows your app to focus solely on its main tasks.

Middleware vs. Routers: What’s the Difference?

Routers: Routers are like traffic directors, deciding where a user’s request should go. Example: A user visits /login, and the router directs the request to the loginController.
Middleware: Middleware acts as a filter or helper that processes requests before they reach the router or handler.

Tasks middleware can handle:

Checking if the user is authenticated.
Logging requests.
Modifying or validating data.
Managing errors.

How They Work Together

Step 1: User Request
Example: A user makes a request to /login.
Step 2: Middleware
Middleware runs first, performing background tasks like authentication or data transformation.
Step 3: Router
Once middleware is done, the router directs the request to the correct handler, such as loginController.
Step 4: Handler/Controller
The handler processes the request and sends the response back to the user.

Simple Analogy

• Middleware: Like a security guard verifying your ID before entry.
• Router: Like a directory showing you the correct room to enter.
• Handler: The room where the actual work gets done.

Middleware ensures your app runs smoothly and securely by taking care of repetitive, behind-the-scenes tasks. It’s an essential part of modern web development that keeps your app focused on delivering its core functionality.

...Happy Codding...

“The role of routers in the MVC pattern”

shema — Thu, 26 Dec 2024 16:56:14 +0000

An often overlooked yet vital skill for mastering the MVC pattern or similar architectures is understanding their core principles. Here, we demystify the concept and break it down in a clear, easy-to-follow way:

MVC Pattern Overview

Model: Handles the data and business logic (Example: Fetching or storing data in the database).
View: Displays the data to the user (Example: HTML, templates, or the user interface).
Controller: Acts as the “middleman,” coordinating between the Model and the View.
But… how does the app know which controller to call when the user interacts with it? That’s where routes (or the router) come in.

Curious to know? Let’s demystify,…

What is the Router’s Role?

The router is like a traffic cop. It directs requests from users to the correct controller function based on the URL they are trying to access. For example:

If a user visits yourapp.com/login, the router ensures the Login Controller is triggered.
If they go to yourapp.com/profile, the router activates the Profile Controller.

“The role of routers in the MVC pattern”

shema — Thu, 26 Dec 2024 16:49:30 +0000

MVC Pattern Overview

Curious to know? Let’s demystify,…

What is the Router’s Role?

The router is like a traffic cop. It directs requests from users to the correct controller function based on the URL they are trying to access. For example:

If a user visits yourapp.com/login, the router ensures the Login Controller is triggered.
If they go to yourapp.com/profile, the router activates the Profile Controller.

How JavaScript works underneath the hood?

shema — Mon, 23 Dec 2024 07:28:48 +0000

This post really aims to cover in a simple way how JavaScript works underneath the hood so that even a new programmer would be capable to grasp the concept and visualize what happens when coding JavaScript.
To start off, there is at least 3 questions I wanted to focus on that will facilitate to overcome difficulties and internalize the logic behind?
These questions are also ones that are likely to be asked during job interviews for web developers where JavaScript implies:

1. How does JavaScript works?
2. Explain the difference between Synchronous vs Asynchronous?
3. Or explain this statement: JavaScript is a single threaded language that can be non –blocking?

Really, It is not necessary to know how JavaScript works internally to write a program but it is essential and crucial to learn in order to understand what is happening behind and feel what you are writing hence some of many developers who have many years in carrier don’t care to know this.

Lets first know what is a program? A program is simply a set of instructions that tells a computer what to do and how to execute the task. The program must allocate the memory, otherwise, we won’t be capable to have variables or even save files on our computers. The program should also parse (read) and execute a dedicated task and all happens in a memory.

Now, JavaScript has engine known as JavaScript engine that each browser implements. For instance in Chrome it is called V8, in Mozilla Firefox: Spider Monkey, Safari browser: JavaScript Core Webkit.

The following image shows the google chrome's V8 engine

What happens inside JavaScript Engine?

JavaScript engine like V8 in chrome reads JavaScript codes that we write and convert them into machine executable instructions for the browser. The above figure is showing parts of JavaScript engine that consists of two parts namely A memory heap **and **A call stack.

It is also important to note that the memory allocation happens in memory heap, while parsing (reading) and execution take place in Call Stack. In addition to this, it is the memory heap that tells you where you are in the program.

Let’s see memory allocation in memory heap with JS (JavaScript) codes

const a = 4;  // now we allocated a memory. JS engine is going to remember
// that a has a value of 4. 

const Obj = {a, b, c };  // In memory, variable 'Obj' holds the object {a, b,c}

// The same as on array. the engine will remember values of the array
const Array = [1,2,3,4,5]

So, what is an issue with the above code once declared globally?

There is something called Memory Leak. As it was mentioned above, the variable declaration happens in memory heap and it has a limited size to be allocated. When you keep on declaring global variables that are very big arrays instead of numbers and even unused, this fills up the memory and results to the Memory Leak. You will hear that global variables are bad due to when we forget to clean up we fill up this memory heap and eventually the browser won’t be able to work.

What about the Call Stack?

If we remember, it is the Call Stack that reads and execute scripts. Let’s use codes to make it clear.

// Example Call Stak

console.log("x");
console.log("y");
console.log("z");

// Result in browser

// x
// y
// z

With the above codes, the call sack reads the first line console.log(“x”); and gets put into the call stack, the JavaScript engine recognizes that the console.log has been added, then pop it into the call stack, runs it, and outputs x. After, it removes the first console.log as it has finished to run it and place the into the second console.log(“y”), adds it on the Call Stack, execute y and removes the second console.log. Finally gets console.log(“z”) using the same process.

This is the simplest demo, what if a bit more of a complex example? Let’s give a typical example:

// Demo Example

const example1 = () => {
  const example2 = () => {
    console.log('7');
}
example2();

// Result:

// 4

Now, what happened in the above code according to the call stack? Let’s see how it will run the above block of code :

//CALL STACK

Function example1() will ran first, then function example2() comes on top of the call stack and gets ran which prints out number 7 as output after checking if there is other codes to run. After this, it will start to remove from the call stack in order starting by console.log(‘7’), example2(), example1() and the call stack is now empty.

> Do we remember this statement? JavaScript is a single threaded language that can be non–blocking.

Single threaded means that it has only one call stack. It can perform only one thing at a time and it is important to emphasize that the Call Stack is First In Last Out like a Stack.

Other languages can have many call stacks what is called multi-threaded which could be more advantageous to have multiple call stack so that we don’t keep waiting around for tasks.

> But, why was JavaScript designed to be single threaded though?

To answer this question, normally running code on a single thread can be pretty much easy since there is no complicated scenarios that arise in multi-threaded environment. You actually have one thing to care about. In multi-threaded there could be issues like Deadlocks. With this theory, we can easily know what a Synchronous programming means.

Synchronous programming simply means: first line of code get executed, the second follows and the third gets executed etc …

To be more explicit this means that console.log(“y”) can not run until console.log(“x”) finishes and console.log(“z”) doesn’t start until both first two finish because it is a Call Stack.

It is likely for programmers to use the site stackoverflow.com. what does the name mean? Well. let’s see:

How Stack overflow occurs

The above image shows how a memory leak can happen and how the memory heap of a JavaScript engine can overflow. Here the call stack receives many inputs greater that it’s size and overflows.

It is possible to demonstrate the stack overflow with the help of code:

// A recursion function that creates a stackoverflow

function foo(){
  foo();
}
foo();

// This function will keep looping over and over and keeps adding foo() to
// the call stack and ends up to the stackoverflow.

// Need to have fun? run this in the browser to see.

Mind that JavaScript is a single threaded only one statement is executed at a time . Here is a problem now: what if console.log(“y”) in the following block of code has a big task that will take longer to be executed for example looping through an array that has thousands or millions of items? what would happen there?

// Example Call Stak

console.log("x"); 
console.log("y");
console.log("z");

// Result in browser

// x
// y

The first line will execute, and assume the second line has a massive job to perform, therefore the third line will wait a long period to get executed. In the above example this doesn’t mean much but let’s think of a big website that perform heavy operations, user would not be able to do anything. The website will freeze until the task is done and the user wait there. This is a bad experience when comes to the performance.

Well, with synchronous tasks, if we have one function that takes much time, it is going to hold up the line. So, it sounds like we need something non-blocking. Remember the statement I mentioned above: JavaScript is a single threaded language that can be non –blocking.

Ideally, in JavaScript we don’t wait around for things that take time. So, how do we go around this problem?

As a rescue, there is Asynchronous programming. So, what is this?

Think of asynchronous like a behavior. Synchronous execution is great because it is predictable. In synchronous we know what happen first, what happen next etc, but it can gets slow.

When we have to do things like image processing or making requests over the network like API calls, we use something more than just synchronous tasks that is Asynchronous.

Let’s see how we can do asynchronous programming with codes:

console.log("x"); 
setTimeout(() =>{
  console.log("y");
},3000);

console.log("z");

// Result

      x
      z
      y   // comes after 3 seconds latter

Now, basing on the above codes it seems like we skipped the second line and execute the third and wait on 3 seconds to output the result. that is asynchronous happening.

In order to understand this and what happened let’s use the following figure.

JavaScript Run-Time Environment

For JavaScript to run, we need more than Memory Heap and Call Stack. we need what is called JavaScript Run-Time which is part of the browser. it is included in browsers. On top of the engine, there is something called web APIs, Callback queue and Event loop as shown on figure.

let’s discuss on the code now where we use setTimeout function.

console.log("x"); 
setTimeout(() =>{
  console.log("y");
},3000);

console.log("z");

The setTimeout function is part of web API and not part of JavaScript, instead it is what browsers give us to use to enable us to do asynchronous programming. so, let’s provide more details for clarification.

CALL STACK: The console.log(“x”) goes into the call stack , runs and we console.log to the browser. After that, the setTimeout(() =>{console.log(“y”);},3000); goes into the call stack because the first task is finished then go to the second one.

Now, here there is a thing, while reading the code, the call stack will detect that there is a setTimeout function that has been set and it is not part of JavaScript but part of web API (See the figure JavaScript Run-Time Environment ) and has it has special characteristic. What happens is that setTimeout triggers the WEB API and because the web API is notified, the function will be popped out of the call stack.

Now, the web API starts a timer of three seconds knowing that in 3 seconds it has to do the task. Remember here, because the call stack is empty, JavaScript engine continues to line 3 which is console.log(“z”); and execute it. That’s why we got result x,z but we have setTimeout three seconds in web API. then, after three seconds when the time limit is up, setTimeout runs and sees what is inside of it and it is done. Once it’s done, the web API will recognizes that it has a callback() function of setTimeout and adds it to the CALLBACK QUEUE being ready to run it.

We come to the last part which is** EVENT LOOP*. This one keeps checking all the time if the call stack is empty. When it is empty and nothing that is currently running in the JavaScript engine, it will check the callback queue and find our **callback()* function with console.log(“z”) then place it in CALL STACK and gets run. Once it’s done, pops it out of the call Stack. Now everything is empty and get the result x z y.

Conclusion: In this post we have seen a lot of information about what happen underneath the hood to completely understand JavaScript logics both tasks that are performed synchronously and Asynchronously.

Hopefully this will help new and advanced JavaScript programmers to enjoy coding in JavaScript related frameworks like ReactJS or AngularJS becouse this is the foundation to start on when understanding advanced logic.

> Happy Coding

References

https://www.freecodecamp.org/news/how-javascript-works-behind-the-scenes.
https://www.simplilearn.com/tutorials/javascript-tutorial/callback-function-in-javascript#

“In the MVC design pattern, why is the router indispensable? What role does it play?"

shema — Wed, 18 Dec 2024 09:42:59 +0000

MVC Pattern Overview

Model: **Handles the data and business logic (Example: Fetching or storing data in the database).
**View: Displays the data to the user (Example: HTML, templates, or the user interface).
Controller: Acts as the “middleman,” coordinating between the Model and the View.

But… how does the app know which controller to call when the user interacts with it? That’s where routes (or the router) come in.

Curious to know? Let’s demystify,…

What is the Router’s Role?

The router is like a traffic cop. It directs requests from users to the correct controller function based on the URL they are trying to access. For example:

If a user visits yourapp.com/login, the router ensures the Login Controller is triggered.

If they go to yourapp.com/profile, the router activates the Profile Controller.

Without routes, your application wouldn’t know what logic to execute for a given user action or request.

Why is the Router Crucially Important?

From our knowledge up to now, can we at least imagine its importance?

1. Organized Request Handling:

o Routes create a clear map of your app’s behavior.

o Instead of cramming logic everywhere, you assign responsibilities like:

/users → Handle user actions.

/products → Manage product actions.

2. Separation of Concerns:

o Routing separates URL-to-controller logic from the rest of your code.

o This keeps your application modular and easier to maintain.

3. Dynamic Behavior:

o Routes let you handle dynamic parameters. For example:

/users/:id → Can dynamically fetch user data based on their unique id.

4. Middleware Integration:

o Routers allow you to easily plug in middleware for specific routes. Example:

Add authentication checks only to protected routes like /dashboard.

5. Scalability:

o In larger apps, the router lets you divide routes into modules (Example: user-related routes, admin routes, etc…).

o This makes scaling your app’s logic a breeze.

How It Fits in MVC

Think of the router as the connection point:

A user makes a request (Example: GET /products/123).
The router checks its list of rules and determines:
Which controller to call.
What functionality to execute in that controller (Example: fetching product with ID 123).

The controller then uses the Model to get the data and passes it to the View for display.

Sample Example to have a strong understanding

Let’s use Node’s Express.js

Router (routes/products.js):

const express = require('express');
const router = express.Router(); // Invoking express's router library
const productController = require('../controllers/productController'); // Importing the controller routes

// Define a route for fetching a product by ID
router.get('/:id', productController.getProductById);

// Exporting routes to be eccessible in app.js
module.exports = router;

Controller (controllers/productController.js):

exports.getProductById = (req, res) => {
  const productId = req.params.id;
  // Call the Model to get product data. Notice here I'm using NoSQL(MongoDb) querying language.
  const product = Product.findById(productId);
  res.json(product); // Send product data to the user
};

Application (app.js): The first application entry

const express = require('express');
const app = express();
const productRoutes = require('./routes/products');

// The route to be triggered when user requests a product
app.use('/products', productRoutes);

app.listen(3000, () => console.log('Server running on port 3000'));

When a user requests a specific product for instance, the first entry of the application which is app.js calls the defined routes(here productRoutes). The productRoutes function hundler calls the relevant function in routes/products.js file (here getProductById) which is imported from the controllers (here controllers/productController.js) folder but assigned to productController constant. The function in controller calls the model to get product data and ultimatly update the view with the data. Note that it’s the controller(the middleman) that updates the view with the fetched data.

In Summary

The router is essential because it:

Decides which controller to activate based on the URL.
Keeps your code modular, clean, and scalable.
Adds flexibility with dynamic paths, middleware, and custom behavior.

Without a router, your app would be chaotic and nearly impossible to scale as you add more features!

Thanks and…Happy coding…