I remember the first time I discovered cron. I’ve been obsessed with automation my whole life, and as a fledgling developer cron became my workhorse.
At the time, I worked at an IT help desk. I used cron for email reminders, automated reports, backups: anything I could schedule, I did.
cron is simple: you just tell it what you want to run, when you want to run it:
0 0 * * * node script.js # run script.js once a day
But it comes with tradeoffs: primarily, you have to operate a server to run it, which costs money and time. If you don’t already have a server where you can run jobs, setting one up just for cron is far from ideal.
In the era of the cloud, it’s also outdated. Cron jobs are simple only if you cut your teeth on Linux and can effectively administer a server. This is an inaccessible option for many modern devs who operate far up the stack.
So it’s no surprise that we have other choices for scheduling code in 2020:
These options work well in their intended context: Lambda is great when you’re glueing together AWS resources. Kubernetes CronJobs or ECS Tasks are ideal when your code runs in a container. Airflow is great when you have complex dependencies between jobs, for example in data pipelines.
But when you just want to run a script on a schedule, they’re overkill.
Enter Pipedream
Pipedream is an integration platform for developers. You run serverless workflows - any Node.js code - triggered by HTTP requests, timers, emails, and more. You can also use pre-built actions to connect to hundreds of APIs and apps - actions are just Node functions that perform common operations against these APIs.
You can sign up and run any Node code on a schedule in less than one minute. Watch this video or follow the step-by-step instructions below:
First, install the Pipedream CLI:
curl https://cli.pipedream.com/install | sh
cd into a directory with a Node script you’d like to run, or just create a simple one-line script:
echo 'console.log("Hello, world")' > cronjob.js
then deploy that script to Pipedream, running it every 15 seconds:
pd deploy --run cronjob.js --timer --frequency 15s
This step will prompt you to sign up for Pipedream if you haven’t already. The pd CLI will deploy your code to Pipedream, and print logs as the script produces them. You can press Ctrl-C to quit the real-time stream, and listen for new logs later by running:
pd logs cronjob-js
You can delete this job and all its logs by running:
pd delete cronjob-js
You can list all running jobs with pd list, update their code or schedule with pd update, and more. See the docs to learn more.
A practical example: send an HTTP request on a schedule
If you already host code at some URL, and just want to trigger a job to run via HTTP request, you can use Pipedream to send that request on a schedule. In this example, we’ll hit the Star Wars API once a day.
First, open your editor and create a file called http.js with the following contents:
const axios = require("axios");
const resp = await axios({
method: "GET",
url: `https://swapi.co/api/films/` // replace with your URL
});
console.log(resp.data);
Then run
pd deploy --run cronjob.js --timer --cron “0 0 * * *”
This will make a GET request to SWAPI, once a day at midnight, but you can modify this script to send any HTTP request, at any schedule. See our guide on making HTTP requests in Node for more examples.
See the README on cron jobs or the Pipedream docs to learn more about the platform at large.
We ❤️ Feedback
We're eager to hear your feedback. We're shipping new features every day - check out our roadmap and reach out anytime via email or Slack to let us know what we can improve.
Top comments (0)