DEV Community: Ayanabilothman

NestJS Main Compnents in One Image🤩

Ayanabilothman — Thu, 17 Oct 2024 13:06:28 +0000

res.cloudinary.com

File-Type Validation in Multer is NOT SAFE🙃

Ayanabilothman — Wed, 02 Oct 2024 07:41:00 +0000

When dealing with file uploads in Node.js, one of the most used packages is Multer. It’s simple to use, integrates well with Express, and allows for straightforward file handling. But unfortunately, Multer’s file-type validation is not secure enough 😓. By default, Multer does not validate the actual content of a file; it only depends on the file extension or the MIME type provided by the client, which is editable. The client can easily change the extension of the file in his operating system. This can lead to serious security vulnerabilities in your application.

The problem😕

When a user uploads a file, Multer allows you to check the file's MIME type or extension to decide whether to accept it using the fileFilter option passed to multer. Here’s a typical way to use Multer's fileFilter to check file types:

import multer from 'multer';

const fileFilter = (req, file, cb) => {
  if (file.mimetype === 'image/png' || file.mimetype === 'image/jpeg') {
    cb(null, true);
  } else {
    cb(new Error('Invalid file type, only PNG and JPEG is allowed!'), false);
  }
};

const upload = multer({
  dest: 'uploads',
  fileFilter,
});

In the above example, Multer checks the MIME type of the file to ensure it's either a PNG or JPEG. This sounds pretty safe, right? Wrong 😏. The MIME type that Multer checks is provided by the client, which means it's extremely easy to manipulate.

For example, a malicious user can rename a .exe file to .jpg and modify its MIME type to image/jpeg. Multer will happily accept this file, assuming it’s a valid image,🙄 but in reality, the user has uploaded a potentially dangerous file.

The Inaccuracy of MIME Type Validation 👎

MIME types are just metadata—they don’t represent the actual content of the file. A determined attacker can easily bypass this by changing a file’s extension or setting the wrong MIME type header in their request.

The following scenario can happen:

A user renames a harmful file virus.exe to image.jpg.
Then he sends the file to the server with the MIME type image/jpeg.
Multer accepts this file based on the MIME type.

This is why depending only on MIME type validation becomes dangerous.

The Solution 🤗: Validate Using Magic Numbers (The file signature)

To accurately verify the content of a file, you need to inspect its magic number—the unique binary signature that identifies the file type. Magic numbers are stored at the beginning of a file and are much harder to change than MIME types or extensions.

You can check for the magic number by manually writing some code on your own, but fortunately there’s a great package for this in Node.js: file-type. It helps determine the actual file type by reading the first few bytes of the file, ensuring the file is genuinely what it claims to be.

file-type package has a method fileTypeFromBuffer that identifies the type of the file using the buffer format of the file. Since in the above code snippet, we store the file in the disk storage, we need a way to convert this file to buffer. This can be done easily using fs built-in package in Node.js.

You can implement express middleware to truly validate the file using the following code:

import { fileTypeFromBuffer } from "file-type";
import fs from "fs";

// Middleware to validate file type by magic number (file signatures)
export const fileValidation = async (req, res, next) => {
  try {
    // get the file path
    const filePath = req.file.path;
    // read the file and return buffer
    const buffer = fs.readFileSync(filePath);
    // get the file type
    const type = await fileTypeFromBuffer(buffer);
    // validate
    const allowedTypes = ["image/jpeg", "image/png"];
    if (!type || !allowedTypes.includes(type.mime))
      return next(new Error("Invalid file type"));

    return next();
  } catch (error) {
    return next(new Error("Internal server error"));
  }
};

Hint
If you store your file in the memory, instead of disk storage, you can access the buffer data directly from req.file.buffer instead of using readFileSync

Why it is important? 🧐

Security: Malicious users can easily bypass Multer’s built-in validation by renaming files and changing their MIME types. Using magic number validation ensures that you’re accepting only legitimate file types. If you’re dealing with sensitive data, ensuring file integrity is critical.
Reliability: While MIME type headers can be altered by users, magic numbers are tied to the file’s content and cannot be easily faked.

So by inspecting the actual content of the file, you can prevent users from uploading dangerous files disguised as harmless ones, making your application safer and more secure.

Redis caching with Mongoose

Ayanabilothman — Tue, 01 Oct 2024 21:59:08 +0000

If you are working with NoSQL databases and use the Mongoose package as an ODM to execute the queries on the database. This article is for you.

Why caching?

Caching is essential in optimizing performance, improving scalability, and enhancing the user experience of modern applications. It can significantly reduce the load on backend resources such as databases, or APIs. By serving cached data instead of executing resource-intensive operations.

How can we apply caching with Mongoose?

mongoose object imported from the mongoose package has a powerful other object called Query. We can add to its prototype any customized method. We will take advantage of this by creating a cache method, to be able to apply it on any query we need to cache its results.

As shown in the following code:

import mongoose from "mongoose";

mongoose.Query.prototype.cache = async function () {
  // generate unique redis key
  const filterObj = this._conditions;
  const options = this.options;
  const modelName = this.mongooseCollection.name;
  const findMethod = this.op;
  const redisKey = JSON.stringify({
    ...filterObj,
    ...options,
    ...{ model: modelName },
    ...{ method: findMethod },
  });

  // check if the result is chached before
  const cached = await client.hGet(modelName, redisKey);

  if (!cached) {
    const result = await mongoose.Query.prototype.exec.apply(this, arguments);
    client.hSet(modelName, redisKey, JSON.stringify(result));
    return result;
  }

  // cache the results
  const cachedResults = JSON.parse(cached);
  return Array.isArray(cachedResults)
    ? cachedResults.map((doc) => this.model.hydrate(doc))
    : this.model.hydrate(cachedResults);
};

app.get("/user", async (req, res) => {
  const users = await User.findOne({ age: { $gt: 30 } }).cache();
  return res.json({ results: users });
});

In this code, we create a cache method that easily can be applied to any query. The logic of this method is simple :

1- generate a unique key name.

2- check if this key has a value cached before.

3- if yes, return the cached result.

4- if no, cache the query's result and return it.

Knowing that, the values are saved in a hash table, to optimize the performance.

With Query prototype we can add paginate method, for instance, or any other customized method to be applied to a query and write more clean code.

DO NOT use 'skip' to paginate in mongoose 🫣

Ayanabilothman — Mon, 17 Jul 2023 11:07:11 +0000

In this post, we won't talk about pagination definition, although we will go through how to perform pagination with another approach. It's preferred to have a background of what index is in databases.

In usual, there is no web application without a pagination feature. As it will be madness if we decide to display all the documents existing in the database on one single page! 😅

And the reason behind this is the cost of reading all documents all at once from the database, and send this duty heavy payload through the response.

Usually, the following code is used to perform pagination using mongoose in nodejs

await User.find().skip(0).limit(10)

This statement will query the users in the User model and select the first 10 documents but without skipping any document as the value for skip method is 0.

But what if we need the 'second' 10 documents, i.e. the 11th document to the 20th. We will rewrite the above statement to be 👇

await User.find().skip(10).limit(10)

The cost behind this statment, that it will scan all documents from the beginning to only be able to 'skip' whatever the number you pass as an argument! That means if you need, for instance, to skip the first 100 documents, it will go through all these documents as well to be able to skip them!

As a result, performance decreases as the number you pass increases because using 'skip' will make the query take longer to execute. 😕

So, what is the alternative approach 🤔?

Simply, we can use indexes and work with range queries to prevent scanning unneeded documents.

We can apply this by using a combination between sort method and $lt or $gt operator, but you should pass an indexed field as _id to the sort method .

This approach is called Cursor-based pagination

Cursor-based pagination works by returning a cursor as an ObjectId or a timestamp that represents the position of the last document in the current page. The next page can then be retrieved by passing this cursor to the query, which will return the documents that come after the cursor.
For example 👇

await User.find( { _id: { $gt: currentCursor } } ).sort('_id').limit(10)

And to be accurate this approach also has a downside 😞, what if any of the documents in the User model is updated or deleted! The cursor may not be accurate in the case, and by fault, some documents could be ignored!

So finally, we could say that these are two different approaches to perform pagination, and each one has its downsides, so you should carefully choose which one will be better depending on the case you work on. 😄

Class Decorator in TypeScript 👋

Ayanabilothman — Thu, 25 May 2023 12:45:19 +0000

Welcome 🤗 and first of all, I should mention that this article requires a background on TypeScript, TypeScript Generics and Object Oriented Programming.

Now, what is a decorator? 🤔 In the real life, decorating something means to change its style of or its functionality. Have you ever changed the design of your room or purchased for an extensible sofa that can be converted into a bed 😅?
In the world of programming, a decorator is a design pattern, it is just a function that wraps another function to add or modify some features of it.

function decoratorFunc(fn) {
  //some code
  fn();
  //some other code
}

The above function decoratorFunc is a decorator in its simplest form. You can pass any function and add more code to be executed either before or after the main function fn.

So, _what are the types of decoratorsin TypeScript?
_ 🤔

class decorator
method decorator
accessor decorator
property decorator
parameter decorator

Which means that a decorator function can be attached to any of the above entities.

In this article we will dive deep into class decorator.
To apply decorators they must be prefixed with the @ symbol and placed immediately before the construct to be decorated.

We now know that the decorator is just a function, so to create a Class Decorator in TypeScript we just write a function, but it must have one argument which is the class itself. 🤓

function classDecorator(target: Function) {
  console.log(`This is to decorate class ${target.name}`);
}

This is the simplest class decorator you will ever see 😂.
The target parameter is the class constructor "the class that the decorator will wrap".

So let's create a class to make the example clear.

class User {
  constructor(
    public fName: string,
    public lName: string,
  ) {}

  fullName(): string {
    return `${this.fName} ${this.lName}`;
  }
}

To apply the classDecorator decorator to the User class we should write

@classDecorator
class User {
  constructor(
    public fName: string,
    public lName: string,
  ) {}

  fullName(): string {
    return `${this.fName} ${this.lName}`;
  }
}

Because we prefix classDecorator with @ and place it before the User class , class User will be passed to the classDecorator.

If you try to execute it, you should see the logs we print inside the decorator.
But it is important to know that the decorator will get executed on the runtime not on the class instantiation.

And voila, you create your first class decorator 🥳

What we have just written is a Decorator Function, what if we need to pass another argument to this function?!🤔 and it should take only one which is the class.

In this scenario we will need a Decorator Factory Function again, just a function the only difference that it can take any number of arguments and it returns the decorator function itself 👇

function logCustomMessage(message: string) {
        return function (target: Function) {
            console.log(message)
    }
}

and then we pass the argument on calling it. 👇

@logCustomMessage('This is a custom message !')
class User {
  constructor(
    public fName: string,
    public lName: string,
  ) {}

  fullName(): string {
    return `${this.fName} ${this.lName}`;
  }
}

Hmmm, can a decorator function return values? 🤔
Good question 😅
From the official documentation 👇
If the class decorator returns a value, it will replace the class declaration with the provided constructor function.

A decorator function can only return classes

👉 Use Cases 👈
Let's introduce two examples from which we can take advantage of the decorators and know how to return a class.

Assume we're building a system for a company, so we have structured the main classes, but we need to apply some features to this class that doesn't deeply relate to the logic of the class itself.

Example 1:
We create a Department class that has some members including the code of each department. This code is a unique identifier for each department and must be unique. So, we need to apply a validation feature away from the class logic to handle this case.

The decorator we can write to validate the code. 👇

function uniqueCode<C extends new (...args: any[]) => {}>(target: C) {
  return class Mixin extends target{
    static codes: string[] = [];
    // the following constructor will get executed on creating object of the original class
    constructor(...args: any[]) {
      super(args);
      const code = args[1];
      if (Mixin.codes.includes(code)) {
        throw new Error(`Department code must be unique!`);
      }
      Mixin.codes.push(code);
    }
  };
}

Department class 👇

class Department {
  constructor(public name: string, public code: string) {}
}

And to apply the decorator on the class 👇

@uniqueCode
class Department {
  constructor(public name: string, public code: string) {}
}

The uniqueCode decorator piece by piece 👇

The function definition: we specify type of the argument target, that it will be custom type C which is restricted to new (...args: any[]) => {}. This means that it should be a constructor function that we call using new keyword, and this constructor function can have any numbers of arguments, then after calling it, it will return finally an object.
The function body: we just return a new class that extends the original class, to add more features, and there is the place we implement the validation feature.

Now if you try to execute 👇

const hrDep = new Department("HR", "H259");
const marketingDep = new Department("Marketing", "H259");

it will throw an error.

NOTE THAT the class we returned from the decorator will replace the original one. Try to execute console.log(Department.toString()) before and after applying the decorator to see the difference.

Example 2:
What if we need to reuse this decorator to be applied to the Employee class as well 🤔.
The dynamic information we need is to know the index of the code attribute in each class we will apply the decorator to.

By rewriting uniqueCode decorator and using Decorator Factory Function instead, we can pass extra arguments.

function uniqueCode(codeIndex: number) {
  return function <C extends new (...args: any[]) => {}>(target: C) {
    return class Mixin extends target {
      static codes: string[] = [];
      constructor(...args: any[]) {
        super(args);
        const code = args[codeIndex];
        if (Mixin.codes.includes(code)) {
          throw new Error(`${target.name} code must be unique!`);
        }
        Mixin.codes.push(code);
      }
    };
  };
}

Now we can pass the index of the code attribute according to its definition in the original class.

Applying the dynamic decorator to any class that has code attribute 👇

// 1 is the index of code in the constructor parameters
@uniqueCode(1) 
class Department {
  constructor(public name: string, public code: string) {}

  info() {
    return {
      name: this.name,
      code: this.code,
    };
  }
}

// 2 is the index of code in the constructor parameters
@uniqueCode(2) 
class Employee {
  constructor(
    public name: string,
    public salary: number,
    public code: string
  ) {}
}

And finally we are at the end 😄. You can implement great ideas using decorators and be safe away from the original logic of the class. Try it now 🙂

JavaScript promises are simple 👌

Ayanabilothman — Sat, 11 Feb 2023 21:18:43 +0000

This article will be simple and only mentions basics related to promises in JavaScript and why we need them. Be ready and just read this story and enjoy understanding promises. 🤗.

👀Important to know that:

JavaScript is a non-blocking language. What does this mean? When JavaScript engine starts to run the code, it doesn't implement the statements in the order as you write them 🤔 "in some cases". So let's see how can this look like.

👉🏼 After executing the above code the result will be:
I'm the first statement
I'm the third statement
I'm the second statement

And the reason behind this behavior is the usage of setTimeout function which is an asynchronous function in JavaScript.
So, what is asynchronous function? 🤔

Simply, it's a function that may take some time. Thus, JavaScript engine holds the execution of it and runs the rest of the code. After finishing all non-asynchronous statements, it finally executes this postponed asynchronous function.

This is the exact meaning of non-blocking language, this behavior of holding on the execution of some code written at certain order and be able to execute it later.

Problem definition😓:

Why this non-blocking behavior can be an issue in some cases? What if you need to do some operations ("b" & "c") which depends on another operation "a" that can take some time? 👇

Now let me introduce an interesting case study 🤓.

We need to fetch a video from a database, but this operation can take some time due to internet connection or for any other reasons. After receiving the video, we need to trim it. The last thing we need is to display the trimmed video.

So now we have three operations:
a- Request the database to fetch the video.
b- Trim the received video.
c- Display the trimmed video.

As you see, the three operations depend on each other, you can't trim or display the video until you receive the original one.

Let's define the functions to do each single operation:

Hint: we use setTimeout function inside fetchDB function to just simulate as if we are fetching data from a database.

It is time to execute these operations:

👉🏼 After executing the above code the result will be:
undefined After trim. And here is the final video to display!

Notice that the first word is "undefined". This value is the result of invoked fetchDB function in statement number 20. But why it is "undefined"? because of the same reason fetchDB function has setTimeout function which will take some time. So the engine skips it and in turns it will return nothing, so the value will be undefined.

Since all other functions trimVideo and displayVideodepend on the original video so they won't return the expected result.

What is the solution then?

Solution 1 🙌: Callback functions

To handle this, we can use callback functions. What is a callback function? It's a function that is a parameter of another function.

fn_1 is a simple regular function. What if it takes a parameter? this parameter can be of any type even if it was a function.
How can you invoke fn_1 function? You need to pass the function as an argument.

👉🏼 After executing the above code the result will be:
f1
f2

So now returning to our case, how you use callback functions to successfully execute all operations related to the required video?

The trimVideo function relies on the fetchDB function. We can rewrite fetchDB function to take function as an argument (callback function) which will be the trimVideo function.

In addition, the displayVideo function relies on the trimVideo function.Thus, you can rewrite trimVideo function to take function as an argument (callback function) which will be the displayVideo function.

The displayVideo function will still the same as it is the final operation and there is no more operations depends on it.

Now how can we redefine the main function?

👉🏼 After executing the above code the result will be:
Here is the video. After trim. And here is the final video to display!

Finally, we got the expected results. But it seems that this is a complex code, doesn't it? What we did here is when we invoke the functions, each function that takes callback function added complexity to the code. And this is known as callback hell. Writing nested callback functions is so hard and complex.
More details on callback hell can be found here.

Are there any alternative solutions to avoid using callback functions and simplify the code writing? Yes, thanks to ES6.

Solution 2 🙌: Promises

I know you read a lot to reach here, but the reason behind using promises will help you well understand why promises are so valuable.

What is a promise? Promise is a special object in JavaScript, it has two main attributes or properties (state + result) (Remember this for later discussion) . Let's dive deep and enjoy coding with promises.

As shown in the above code, to construct a promise you need to create an instance object from a predefined class Promise using new keyword. Writing new Promise() will create a promise object. But what is the function inside those round brackets?

(resolve, reject)=>{} is a function which has two arguments (resolve, reject), you can rename them anything else, but by convention they're called resolve and reject.

And now what are those resolve and reject? 🤔 They are two callback functions "but don't worry we won't write here anymore callback functions😅. We just need to understand what we write while constructing a promise". As you knew before, when you write a function as an argument you can call this function whenever you want inside the parent function.

And this exactly what happens. When should you call these functions?

you should invoke the resolve function when you need to return a value if your code has executed in an expected manner.
Otherwise, you should invoke the reject function to return a value if your code failed or an error occurred.

Knowing that, only one function got invoked. If you try to invoke both functions after each other without any conditions. Only the first one will be invoked either it was the resolve function or the reject function.

At the beginning of this section, we say that promise has two main attributes or properties** (state + result)**.

Simply the state can be one of three states:

pending
fulfilled
rejected

While the result is any value of any type which you pass to the resolve or the reject functions when you invoke them.

Results depend on the states. if the state was fulfilled, resolve function get invoked with the result. And when the state was rejected, reject function get invoked with the result.

But _what is a pending state? 🤔 It is the state of the promise which hasn’t been fulfilled nor rejected yet. This state has no results you can deal with.

Ok, you now understand what is promise. The final question, how can it help you?

Any promise object has then property. then is just a function that takes another function as an argument.

The interesting here, that the arguments of this callback function is the result you previously sent through resolve function.

In addition, any promise object has catch property. catch is just a function that takes another function as an argument.

The interesting here as well, that the arguments of this callback function is the result you previously sent through reject function.

then and catch will never get invoked until the promise is fulfilled or rejected meaning that the resolve or the reject function must be invoked.

The last thing you need to know is that then function returns implicit promise by default. Implicit promise is a promise you didn't completely write, when you return any value 'X' from the callback function of then function, this value 'X' acts as a resolved result from a hidden resolve function. Thus, you can write a chain of then functions.

Returning to our problem, how can we use promises to handle our situation?

JavaScript engine deals with this chain of then functions as one unit and doesn't execute any of them until the previous then function resolved. Which means that any code written inside this chain will be executed in the right order as you write and this is what we need.
In addition, the engine will execute those functions in asynchronous way. It won't block any code after then functions.

👉🏼 After executing the above code the result will be:
First statement out of promise
Second statement out of promise
4

Applying these concepts in our operations we can rewrite our code as the following 👇:

The difference between writing the code using promises and writing the code using the callback functions is so obvious. The code is more clear, simple and understandable now 🥳.

We can enhance the writing of promises and chains in some cases using async/await keywords. 👉 Write promises chaining using async/await 👈 article will clarify the idea 😊.

Error handling of promises is a great topic. It will be clarified in another article, just wait for it 🔥.

Write promises chaining using async/await.

Ayanabilothman — Sat, 11 Feb 2023 21:17:09 +0000

First of all, if you don't know anything about JavaScript promises, go through 👉 JavaScript promises are simple 👈 article and enjoy code writing using promises 🤗

async and await are JavaScript keywords that enable us to work with promises in a more simple way.

async

If you define a function that returns a promise, and this promise will always only resolve, in this case, instead of constructing a promise, you can use async keyword before the function and directly return the resolved value.

Let's clarify by an example 👇

You can replace the second function with the first one. Using async keyword before the function will make this function return an implicit promise behind the scenes. The value you actually return from this function will act as if it is a result passed through hidden resolve function.

And in turns, we can use chaining on the returned promise.

What about await keyword ?

await

await keyword is used instead of then function, which means we can let go of writing chains.

Be careful about 🤓

await keyword must be used before promises, you can't await for anything expect promise.
await keyword must be used inside a function with async keyword.

Summary:

async replaces constructing promises.
await replaces then.
async can be used alone, but await must be used inside async function.

Error handling of promises is a great topic. It will be clarified in another article, just wait for it 🔥.