DEV Community: yashvant vala

Polyfills for Array methods: forEach(), map(), filter(), reduce(),find()

yashvant vala — Wed, 22 Jun 2022 15:48:09 +0000

It's been a long time since I haven't written any articles but here we go.

Note: if you don't know what polyfills are then please read this article.

forEach()

Here is the sample code for how forEach will work.

const person = [
  {
    id:1,
    name:'user1',
    isActive: true,
  },
  {
    id:2,
    name:'user2',
    isActive: true,
  }, 
  {
    id:3,
    name:'user3',
    isActive: true,
  }, 
  {
    id:4,
    name:'user4',
    isActive: true,
  }  
]

person.forEach(data=>{
  data.isActive = false;
})

console.log(person)

Output:

[
  { id: 1, name: 'user1', isActive: false },
  { id: 2, name: 'user2', isActive: false },
  { id: 3, name: 'user3', isActive: false },
  { id: 4, name: 'user4', isActive: false }
]

Here I am manipulating the isActive to false. so how can we create our own method like this, to know that see the below example

const person = [
  {
    id:1,
    name:'user1',
    isActive: true,
  },
  {
    id:2,
    name:'user2',
    isActive: true,
  }, 
  {
    id:3,
    name:'user3',
    isActive: true,
  }, 
  {
    id:4,
    name:'user4',
    isActive: true,
  }  
]
Array.prototype.myForEach = function(callback){
  for(let i=0;i<this.length;i++){
    callback(this[i],i, this);
  }
}

person.myForEach(data=>{
  data.isActive = false;
})

console.log(person)

Output:

[
  { id: 1, name: 'user1', isActive: false },
  { id: 2, name: 'user2', isActive: false },
  { id: 3, name: 'user3', isActive: false },
  { id: 4, name: 'user4', isActive: false }
]

In the above example, I have named the method as myForEach and added it to the prototype of the array so it can be available for any array data structure. once we add the method we can use the array using this keyword because we are calling the method using the (.) operator with the array, which means the context of this will be the object where we have called the method.

And above example will give the same output as forEach().

Map()

const person = [
  {
    id:1,
    name:'user1',
    isActive: true,
  },
  {
    id:2,
    name:'user2',
    isActive: true,
  }, 
  {
    id:3,
    name:'user3',
    isActive: true,
  }, 
  {
    id:4,
    name:'user4',
    isActive: true,
  }  
]

const newPerson = person.map(data=>{
  return {...data, isActive:false};
})

console.log(newPerson)

In the above code, we are making isActive as false and it will return a new array, unlike forEach. So the output would be something like this.

Output:

[
  { id: 1, name: 'user1', isActive: false },
  { id: 2, name: 'user2', isActive: false },
  { id: 3, name: 'user3', isActive: false },
  { id: 4, name: 'user4', isActive: false }
]

below is the polyfill for the map method,

Array.prototype.myMap = function(callback){
  const newArray = [];
  for(let i=0;i<this.length;i++){
    newArray.push(callback(this[i],i, this));
  }
  return newArray;
}

const newPerson = person.myMap(data=>{
  return {...data, isActive:false};
})

console.log(newPerson)

Output:

[
  { id: 1, name: 'user1', isActive: false },
  { id: 2, name: 'user2', isActive: false },
  { id: 3, name: 'user3', isActive: false },
  { id: 4, name: 'user4', isActive: false }
]

It is very similar to forEach, the difference is just we are returning a new array with the result of the callback function. so the output will be the same.

Filter()

See below example where we are filtering the users based on the isActive parameter.

const person = [
  {
    id:1,
    name:'user1',
    isActive: true,
  },
  {
    id:2,
    name:'user2',
    isActive: false,
  }, 
  {
    id:3,
    name:'user3',
    isActive: true,
  }, 
  {
    id:4,
    name:'user4',
    isActive: false,
  }  
]

const newPerson = person.filter(data=>{
  return data.isActive;
})

console.log(newPerson)

Output:

[
  { id: 1, name: 'user1', isActive: true },
  { id: 3, name: 'user3', isActive: true }
]

it is similar to a map but it will only return those items which match the condition in the return statement. so let's write polyfill for that

Array.prototype.myFilter = function(callback){
  const newArray = [];
  for(let i=0;i<this.length;i++){
    if(callback.call(this, this[i],i, this)){
      newArray.push(this[i]);
    }
  }
  return newArray;
}
const person = [
  {
    id:1,
    name:'user1',
    isActive: true,
  },
  {
    id:2,
    name:'user2',
    isActive: false,
  }, 
  {
    id:3,
    name:'user3',
    isActive: true,
  }, 
  {
    id:4,
    name:'user4',
    isActive: false,
  }  
]

const newPerson = person.myFilter(data=>{
  return data.isActive;
})

console.log(newPerson)

Output:

[
  { id: 1, name: 'user1', isActive: true },
  { id: 3, name: 'user3', isActive: true }
]

Here we are calling the callback using call method to provide the context of an array. when if the condition becomes true then we are pushing the array and we are only returning those items which satisfy the condition.

const person = [
  {
    id:1,
    name:'user1',
    isActive: true,
    balance: 20,
  },
  {
    id:2,
    name:'user2',
    isActive: false,
    balance: 30,
  }, 
  {
    id:3,
    name:'user3',
    isActive: true,
    balance: 40,
  }, 
  {
    id:4,
    name:'user4',
    isActive: false,
    balance: 50,
  }  
]

const totalBalance = person.reduce((accumulator,data)=>{
  return accumulator+=data.balance;
}, 0)

console.log(totalBalance)

Output:

140

Here I've added one more field called balance and I want the total sum of the balance so we can use the reduce method to make the sum out of it. note that the callback function first argument will be the different and we have provided the initial value as 0.

let's see the polyfill for reduce method in the below example.

Array.prototype.myReduce = function(callback, initialValue){
  let accumulator = initialValue===undefined ? undefined: initialValue;
  for(let i=0;i<this.length;i++){
    if(accumulator!==undefined){
      accumulator = callback.call(undefined, accumulator, this[i], i, this)
    }else{
      accumulator = this[i];
    }
  }
  return accumulator;
}
const person = [
  {
    id:1,
    name:'user1',
    isActive: true,
    balance: 20,
  },
  {
    id:2,
    name:'user2',
    isActive: false,
    balance: 30,
  }, 
  {
    id:3,
    name:'user3',
    isActive: true,
    balance: 40,
  }, 
  {
    id:4,
    name:'user4',
    isActive: false,
    balance: 50,
  }  
]

const totalBalance = person.myReduce((accumulator,data)=>{
  return accumulator+=data.balance;
}, 0)

console.log(totalBalance)

Output

140

here we are passing one more parameter which is the initial value, we are calling the callback using call method and assigning the result to the accumulator so we will get the sum out of it or whatever logic we have written inside the callback function.

Note: don't forget to provide the initial value otherwise as we can see in the polyfill it will assign this[i] means the whole object and we have to extract the value out of it.

Find()

find() is very similar to the filter method but here we are only sending one result.

const foundPerson = person.find(data=>{
  return data.name === 'user1';
})

console.log(foundPerson)

Output:

{ id: 1, name: 'user1', isActive: true, balance: 20 }

As we can see the output that it will only return one record.

Array.prototype.myFind = function(callback, initialValue){
  for(let i=0;i<this.length;i++){
    if(callback.call(accumulator, this[i], i, this)){
      return this[i]
    }
  }
  return accumulator;
}
const person = [
  {
    id:1,
    name:'user1',
    isActive: true,
    balance: 20,
  },
  {
    id:2,
    name:'user2',
    isActive: false,
    balance: 30,
  }, 
  {
    id:3,
    name:'user3',
    isActive: true,
    balance: 40,
  }, 
  {
    id:4,
    name:'user4',
    isActive: false,
    balance: 50,
  }  
]

const foundPerson = person.find(data=>{
  return data.name === 'user1';
})

console.log(foundPerson)

Output

{ id: 1, name: 'user1', isActive: true, balance: 20 }

In the polyfill, we are directly returning the record which matches the condition,

Similarly, there is one more method findIndex which is very same as find, this method will return the index of the array record which match the condition, to achieve that in find polyfill you can write in return statement as return i instead of return this[i] and in else block you can return -1.

So this was it for the polyfill, Hope you get some knowledge from here and if any doubt please write me comment, i would be glad to solve your doubt.

More polyfills are coming, stay tuned and see you in the next article!

signing off!

Upload/Download/Delete files on AWS S3 bucket using node.js

yashvant vala — Tue, 14 Dec 2021 04:44:31 +0000

What is aws s3?

Aws s3 stands for simple storage service provided by amazon web service.

Here are some of the use cases of aws s3

Static Website Hosting

you can host any static web site which is built in (react,nextjs,angular, vue, gatsby,remix).
Archive Data
you can archive much of the data with low cost.
Backups (most used)
you can store your backups like db backup every day using cron job.

Also there are some additional advantage of it which you can explore in aws console

Let's understand how s3 works.

In order to use s3, you need to create a bucket where you want to store anything. you can compare it to folder where all files are stored.

Each bucket has unique name and you can create folder/files in the bucket. It stores data as key value pair. We will see in further that how it stores data in key value pair. S3 provides storage limit 0 to 5 TB.

Generally Root user of aws creates Identity and Access Management (IAM) user to provide access of aws with some rules.

IAM user can create bucket, and also can perform some operations on it.

In order to create IAM user, you need to create account on aws

Once you have created account, you can find IAM user creation option. After creating IAM user, create the s3 bucket using s3 option. See the below image.

Once you click on s3 you will find following screen.

Click on the create bucket and fill all the data, You will also need to select rules like permissions and all.

After creating a bucket aws will provide you Access key id and Secret access key.

Save those key in your machine because we will need those keys in order to perform any actions on it.

Now create a new nodejs project using following command. (I assume that you have downloaded node in your machine).

npm init -y

-y is a flag which is used when you want to give yes to all questions which is asked when you create node project.

Install few dependencies to create an app.

npm i express dotenv express-fileupload

Above dependecies are used to create express server and upload files. dotenv file is for storing the private keys provided by aws.

Now create index.js file with following code.

const express = require("express");
const app = express();
const fileUpload = require("express-fileupload");

//express middlewares
app.use(fileUpload({
    useTempFiles:true,
    tempFileDir:"/tmp"
}));



const PORT = process.env.PORT || 4000;
app.listen(PORT,()=>console.log(`server is running on port ${PORT}`));

We will need express file upload middleware to upload files, we also can create own functionality to create temporary folder and store file in folder, once it is uploaded we can delete that folder.

Let's create two folder, Rotues and utils.

In routes folder we will create routes of api and we will send response based on actions, We surely can create controller folder but as of our job can be done using route.

In utils (utility) folder we will create one file which is called s3.js to perform some actions based on s3.

GO to routes folder and create file with .js extension, Here I have created uploadFile.js

Write following thing in that file.


const router = require('express').Router();
const s3 = require("../utils/s3");

router.post("/upload",(req,res)=>{
    const file = req.files.file;
    s3.uploadToS3(file,(error,data)=>{
        if(error){
            return res.send({error:"Something went wrong."});
        }
        return res.send({message:"File uploaded successfully"});
    });
});

module.exports = router;

Here we are having post method, We will get one file from front-end or you can use post man here.

s3.uploadToS3 is the function which we will create in s3.js file to upload file, we will also return call back with data and error.

Important

We will need to install aws sdk dependency to perform action in aws s3

So write following command

npm i aws-sdk

After Installing it , go to s3.js file and import following things.

const path = require('path');
const fs = require('fs');
const fileUpload = require('express-fileupload');
const aws = require('aws-sdk');

Now we will need to configure the aws to create connection between our app and aws.

//configure the aws environment
aws.config.update({
    accessKeyId:process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey:process.env.AWS_SECRET_ACCESS_KEY,
});

Above code is just used to create environment of s3.

now create instance of s3 so we can use all methods of s3.

//initialize s3
const s3 = new aws.S3();

Now we can create our function to upload files which we are calling in our uploadFiles.js

exports.uploadToS3 = (file,next) =>{
    const fileStream = fs.createReadStream(file.tempFilePath);
    console.log(file.tempFilePath)
    const params = {
        Bucket:process.env.AWS_BUCKET_NAME,
        Body:fileStream,
        Key:file.name
    };
    s3.upload(params,(error,data)=>{
        console.log(error,data);
        next(error,data);
    });
};

So we will need some parameters to send to s3, so s3 can know that where to upload file and in which bucket to.

We will send bucket name, Body means the data which we want to upload and key which you can give any name. I will just give the file name as key name.

s3.upload method accept parameters and based on it it return call back function of data and error.

We will send data and error in our callback function, which we are getting in uploadFiles.js

Now you can go to index.js and import following things.

const uploadFiles = require("./routes/uploadFiles");

//you use this as a moddleware of routes

//our base api route will be /api/v1/*
app.use("/api/v1",uploadFiles);

So final index.js will be look like this.

const express = require("express");
const app = express();
const fileUpload = require("express-fileupload");

//express middlewares
app.use(express.json());
app.use(express.urlencoded({extended:true}));
app.use(fileUpload({
    useTempFiles:true,
    tempFileDir:"/tmp"
}));

//all routes
const uploadFiles = require("./routes/uploadFiles");

//route middlewares
app.use("/api/v1",uploadFiles);

//config http server
const PORT = process.env.PORT || 4000;
app.listen(PORT,()=>console.log(`server is running on port ${PORT}`));

Now we can use our api in postman. localhost:4000/api/v1/upload is the url to can api.

See the image below to select configuration to send file.

I have selected image file so it will be uploaded on s3. You Also can verify on s3 whether it is uploaded or not.

Once it is uploaded we will get following metadata by s3.

ETag: 'some random string',
  Location: your url of file,
  key: 'horizontal_tagline_on_white_by_logaster.jpeg',
  Key: 'horizontal_tagline_on_white_by_logaster.jpeg',
  Bucket: your bucket name

We can store key and location in our db so in future we can select or delete anything based on key.

Now we can create download and delete file functionality which is much easier.

Since we need bucket name in all the parameter which we will pass in all methods, I will make and object so we don't need to write it again.

const constantParams = {
    Bucket:process.env.AWS_BUCKET_NAME
}

So now Our s3.js file will look like this.

const path = require('path');
const fs = require('fs');
const fileUpload = require('express-fileupload');
const aws = require('aws-sdk');

//configure the aws environment
aws.config.update({
    accessKeyId:process.env.AWS_ACCESS_KEY_ID,
    secretAccessKey:process.env.AWS_SECRET_ACCESS_KEY,
});

//initialize s3
const s3 = new aws.S3();

//constant params
const constantParams = {
    Bucket:process.env.AWS_BUCKET_NAME
}

//upload file to s3 bucker
exports.uploadToS3 = (file,next) =>{
    const fileStream = fs.createReadStream(file.tempFilePath);

    const params = {
        ...constantParams,
        Body:fileStream,
        Key:file.name
    };
    s3.upload(params,(error,data)=>{
        console.log(error,data);
        next(error,data);
    });
};

//download file from s3 bucket
exports.getFileFromS3 = key =>{
    const downloadParams = {
        Key:key,
       ...constantParams
    };
    return s3.getObject(downloadParams).createReadStream();
};

//delete file from s3 bucker
exports.deleteFileFromS3 = (key,next) =>{
    const deleteParams = {
        Key:key,
        ...constantParams
    };
    s3.deleteObject(deleteParams,(error,data)=>{

        next(error,data);
    });
};

s3.getObject method used when we want to get any file from s3

s3.deleteObject method will delete the file.

As you can see in above code our function for download and delete is expecting the key, which we get by s3 so we can pass that key

Now our uploadFiles.js file will look like this.

const router = require('express').Router();
const s3 = require("../utils/s3");

router.post("/upload",(req,res)=>{
    const file = req.files.file;
    s3.uploadToS3(file,(error,data)=>{
        console.log("commit")
        if(error){
            return res.send({error:"Something went wrong."});
        }
        return res.send({message:"File uploaded successfully"});
    });
});

router.get("/getFile",async (req,res)=>{
    try {
        let fileToSend = await s3.getFileFromS3('horizontal_tagline_on_white_by_logaster.jpeg');
        fileToSend.pipe(res);
    } catch (error) {
        res.send({error:"Server Error"});
    }
});

router.delete("/deleteFile",(req,res)=>{
    s3.deleteFileFromS3('horizontal_tagline_on_white_by_logaster.jpeg',(error,data)=>{
        if(error){
            return res.send({error:"Can not delete file, Please try again later"});
        }
        return res.send({message:"File has been deleted successfully"});
    });
});

module.exports = router;

Now you can call both download and delete api, the end point of both api will be,

download file : localhost:4000/api/v1/getFile/:key

delete file : localhost:4000/api/v1/getFile/:key

I am passing the key directly because we didn't saved it in our db. but in real scenario we will be saving key in db and fetching keys to perform actions.

So this is how you can perform some actions on s3. Trust me it is very easy than it seems and also depends how you want to craft you app.

Hope you get something from this article, If you still confused at any point then please comment below.

Thanks :)

Schedule the DB backup cron job with nodejs

yashvant vala — Mon, 13 Dec 2021 07:08:32 +0000

Job scheduling has always been an important part of any application where we want to make certain tasks automatically like sending emails to the subscribed user or anything like that.

In this article, we will cover that how we can make a cron job that takes DB backup automatically at a given time interval.

First, we will need to create a node project, Create node app with the following command.

npm init -y

The above command will make package.json file and -y will give all question-answer which asked during making a project.

We will be using express.js and to run the cron job we will be using a package called node-cron

npm i express node-cron

The above command will install express and node-cron package.

Let's create a file called index.js and import the following things.

index.js

const { spawn } = require('child_process');
const path = require('path');
const cron = require('node-cron');

Spawn process will help us to run command in a terminal, It also does many things but we only need it to run our command.

Path will be used to join the given path which will be provided along with the directory name.

The following command will dump a mongo DB backup in the format we will provide.

mongodump --db=<your db name> --archive=./<path>.gzip --gzip

//example

mongodump --db=users --archive=./userDBBackup.gzip --gzip

The above command will have a couple of arguments, first will be the command name which is mongodump, the Second one is DB name, the Third one is the path of the archive where we want to store the DB backup and the last one will be the compression which we want to use.

Now we will create a function that will be responsible to make DB backup.

const DB_NAME = 'users';

const BACKUP_PATH= path.join(__dirname, 'public', `${DB_NAME}.gzip`);

const  backUpDB = () => {
    const child = spawn('mongodump',[
        `--db=${DB_NAME}`,
        `--archive=${BACKUP_PATH}`,
        '--gzip'
    ]);
    child.stdout.on('data',(data)=>{
        console.log('stdoup',data)
    })
    child.stderr.on('data',(data)=>{
        console.log('stdoup',Buffer.from(data).toString())
    })
    child.on('error',(err)=>{
        console.log(err);
    })
    child.on('exit',(code,signal)=>{
        if(code){
            console.log('process exit with code', code)
        }else if(signal){
            console.log('process killed with signal',signal)
        }else{
            console.log("back up success");
        }
    })
}

Let's understand the above code.

So path.join method will join the two-path, __dirname will return the current directory of our location, and also we can provide directory name and file name in the argument.

Spawn will execute the command in a terminal, It takes the first argument as a command name and another argument will be an array of all parameters of a command. Here we just wrote the command of DB backup.

Now we will need to handle the output as well as the error which comes from the terminal or from the file itself.

So stdout will return the output if our command has been executed successfully, It will return a callback function and in the argument, it provides the data.

stderr will return the error if the command is not found or something wrong related to the command or terminal.

Now the third thing is the simple error that can occur in our file. We will also check the code and signal if the process is failed due to some issue.

Let's create a job that can run at a certain time interval and we will call our function there.

If you want to learn more about job scheduling then click here, This link is from one of my articles on a cron job.

So following code will run the job at a given time.

// Scheduling the database backup every night
cron.schedule('00 00 00 * * *', () => backUpDB ());

Here schedule method expects arguments which is the time when we want to run the DB backup code.

To learn more about the combination of different times click here.

So the final code will look like this.

const { spawn } = require('child_process');
const path = require('path');
const cron = require('node-cron');

const DB_NAME = 'users';

const BACKUP_PATH= path.join(__dirname, 'public', `${DB_NAME}.gzip`);

const  backUpDB = () => {
    const child = spawn('mongodump',[
        `--db=${DB_NAME}`,
        `--archive=${BACKUP_PATH}`,
        '--gzip'
    ]);
    child.stdout.on('data',(data)=>{
        console.log('stdoup',data)
    })
    child.stderr.on('data',(data)=>{
        console.log('stdoup',Buffer.from(data).toString())
    })
    child.on('error',(err)=>{
        console.log(err);
    })
    child.on('exit',(code,signal)=>{
        if(code){
            console.log('process exit with code', code)
        }else if(signal){
            console.log('process killed with signal',signal)
        }else{
            console.log("back up success");
        }
    })
}

// Scheduling the database backup every night
cron.schedule('00 00 00 * * *', () => backUpDB ());

So this was a simple article on how you can backup the database.

Note - You can backup any database by their corresponding commands, You just need to pass some arguments in the spawn.

Tip - You also can upload backup on any cloud like AWS S3 or google drive, cloudinary, or anything like that. You can check my another article to know more about uploading files in the cloud.https://blog.yashvant.in/uploaddownloaddelete-any-file-on-aws-s3-bucket-using-nodejs

Thanks :)

Host React/Angular/Vue Site on Netlify

yashvant vala — Sun, 12 Dec 2021 13:03:25 +0000

Hosting your own site is really easy, In this article we gonna see how can we host our front end in netlify.

Also hosting concept will be the same whether it is netlify or vercel or heroku. If you learn any of them then you will be able to host your static site anywhere.

So let's start

Step 1:- Create sample website(maybe in react/angular/nextjs) and push it in GitHub

Step 2:- Sign up to Netlify

Once you have singed up you will see the following screen.

Now click on the new site from git button.

After click on it, you will see the following screen.

Step 3 :- Select the platform where you have pushed your code (in our case it will be github)

Once you select the platform you need to authorize the platform to access the repo of your site.

You will see the following screen once you click on github or any button.

You can search/select any private/public repo of your GitHub etc. and can select it.

Once you select repo/project to deploy, you will see below screen.

Now you can select relevent branch of your project and you need to set command to build your project and you have to provide the path of your site build. If you have react project the it will be npm run build and for angular it will be ng build, you also can find build command in package.json or just google it.

You will see below screen after you clicked on deploy site button.

Here my site name will be thirsty-hoover-727570.netlify.app, you can also change the domain name or you can provide custom domain.

You can change domain name in domain setting which will be similar to below image.

So that's it, how easy it was.

Note - If you want to make update in your website you need to push latest code to your repo and you need to redeploy it, you will find the option for redeployment in your project on netlify.

Thanks for reading, let me know in comment section if there is anything which is difficult to understand.