Building a cronjobs server with NodeJs

Note: This article is made for people who are familiar with Nodejs, if you are coming from another language this can be beneficial as well.

What is a cronjob, It's basically a job or a function that will be executed after a certain amount of time aka scheduled.

In the project we are building right now we needed to have scheduled jobs to update certain parameters into our Database. so this is easy, Right? you can just use this npm package , so where is the problem?

The problem

While developing the project we found that some of the jobs are being scheduled but not Executed, why? because when we push new code into our server we have to restart. and every scheduled job into the memory is deleted forever. so what is the solution?

The solution

We have two options here but the core idea is still the same.
The cronjob should run as a stand-alone instance meaning it's independent of our main application.

1- to schedule jobs based on the OS that our server running on which is a Linux distribution. While this solution might work, for now, the problem is we don't have full control over our server and we might remove the whole project into another server in the future.
2- to make a cronjob server and have a record of these jobs in our database

Important Note: I am not going to share the full code in this article, I am just sharing the core idea.

Making the Server.

• first, we need to make a jobs model

a more simplified version of this model will be

   ` time:{
        type: Date,
        required: true
    },
    done:{
        type: Boolean,
        default: false
    },
    type:{
     type: "Whatever you want it to be"
    }
    canceled:{
        type: Boolean,
        default:false
    }`

Ofc you can add or customize that model as you want but keep in mind that time, done, canceled are the most important parameters in this model.

• second install express and mongodb and node-schedule.
• third make a simple server that starts after connecting to The DB.

here is a simple configuration for this

DB.js Config

const mongodb= require('mongodb');
const dbService = {

    db:null,
    connect: async function connection (){
        return new Promise ((resolve,reject)=>{

            mongodb.MongoClient.connect(process.env.MONGODB_URL,{
                useUnifiedTopology:true
            },(err,client)=>{
                if (err){
                    return reject(err);
                }
                this.db = client.db(process.env.DB);
                resolve(true);
            })
        });
    }
}

Now make a server and end-point to receive job requests, And another one to cancel jobs if you want to.

const express = require('express');
const dbConfig = require('./DB');

dbConfig.connect().then(()=>{
    app.listen(5000,()=>{
        console.log("server is listening on port 5000");
    });
   // re-schedule jobs (that are not Done yet) if the server restarts 
   onServerRestart();

}).catch((e)=>{
    console.log("couldn't connect to database Restart the server");
});

• end-point to schedule a job and another to cancel.

router.post('/',(req,res)=>{

    const job = req.body.job;
    // job is a Document that is created in the main application  
   // and sent to this server to be scheduled 

    scheduleJob(job);
    return res.sendStatus(200);
});

router.get('/cancel',(req,res)=>{
  // job id is sent from the main application
   const jobID = req.query.id;

  // this will be explained later
   Emitter.emit(jobID);
   return res.sendStatus(200);
}

inside job schedule function

const sched = require("node-schedule");
// this will also be explained later 
const Emitter = require("../Emitter/cutomEmitter");

async function scheduleJob(job){
   const newJob = sched.scheduleJob(job.time,()=>{
      // do the job after a certain amount of time 
      and now this job is in memory 
   });
}

now, what if you want to cancel the job? node-schedule gives you a way to do that by calling newJob.cancel(). But how will you do that from another server? remember this server is just made to schedule jobs. here comes the Event Emitter API.

refactoring the function to cancel jobs.

async function scheduleJob(job){
   const newJob = sched.scheduleJob(job.time,()=>{
      // do the job after a certain amount of time 
      and now this job is in memory 
   });
   function cancelJob(j){
      j.cancel();
     //access the DB and find Job by ID and cancel it 
        db.db.collection("jobs").updateOne(
            {
                _id: ObjectID(_id),
            },
            {
                $set: {
                    cancelled: true,
                },
            }
    );
   }
// now how the hell are we going to access this function? 
//using emitter -> don't worry i will show u how to configure one 
    Emitter.once(_id, cancelJob);
 // using Emitter.once cause this removes the whole Event which is "_id" -> referring to the job u want to cancel. 
// If the job is executed after a certain amount of "time" then you don't need that event, time to remove it. 
    sched.scheduleJob(job.time,()=>{
         Emitter.removeListener(_id,cancelJob);
   });
}

here is the Emitter.js config

const EventEmitter = require('events');

class jobEmitter extends EventEmitter{}
const Emitter = new jobEmitter();

module.exports = Emitter;

yes, it is that easy.

now let's use our CronServer

Usage

The scenario is in server 1 you need to schedule a job

• first, if you are using mongoose just export the Jobs model and jobs.Create({
time: new Date(anytime that you want)
note you might want to add certain parameters here to specify the job
// which I suggest you do
});

• send a post request to CronServer with the job to be scheduled.

axios.post("http://localhost:5000/,job,{
//might want to config that request
});

• look into your Database to see if the job is scheduled or no.

time to test the cancellation request.

axios.get(http://localhost:5000/cancel?id=<jobID>);

Check if the job is canceled or no you should see in the terminal the console.log(job with ${_id} is canceled);.

• try hitting the same request again you won't get anything because the emitter has been removed and the job has been canceled.

Final Notes

• onServerRestart function is made to reschedule jobs if anything happened and you need to restart the CronServer just search for the jobs that have (done: false, canceled: false) if the time is less than now Date execute them IMMEDIATELY without re-scheduling, else just re-schedule.

if you have any questions you can contact me via [e-mail]. Thanks for coming to my TED talk.