DEV Community

Cover image for DigDag - Schedule a Job(workflow) multiple times on same day
Shubham Kadam
Shubham Kadam

Posted on

5

DigDag - Schedule a Job(workflow) multiple times on same day

Hey folks! So, I had a task of scheduling a Dashboard execution daily at 12.00 pm and 4.00 pm and was supposed to use DigDag Job Scheduler. But, after spending quite some time on DigDag’s documentation, I couldn’t find anything relevant as per my requirement. Hence, I decided to try a workaround.

"Instead of scheduling the Dashboard using a single workflow (the *.dig file), I can import/include the workflow into another workflow(s) and schedule it as per my need."

For example:

Note: The below example is demonstrated on Ubuntu 18.04 (Linux). Please refer DigDag’s documentation for other platform-specific commands.

Lets generate a sample digdag project schedule-demo using command:

$ digdag init schedule-demo

The above command will create a workflow schedule-demo.dig in schedule-demo directory/project with contents:

timezone: UTC


+setup:

 echo>: start ${session_time}


+disp_current_date:

 echo>: ${moment(session_time).utc().format(‘YYYY-MM-DD HH:mm:ss Z’)}


+repeat:

 for_each>:

 order: [first, second, third]

 animal: [dog, cat]

 _do:

 echo>: ${order} ${animal}

 _parallel: true


+teardown:

 echo>: finish ${session_time}

Let's schedule the above workflow for every one and two minutes on same day using file one_minute.dig and two_minutes.dig workflow.

Create workflows one_minute.dig and two_minutes.dig in the project schedule-demo with below-mentioned contents:

  • Workflow one_minute.dig:
!include : ‘schedule-demo.dig’

schedule:

 minutes_interval>: 1
  • Workflow two_minute.dig:
!include : ‘schedule-demo.dig’

schedule:

 minutes_interval>: 2

Now change your current working directory to schedule-demo using the command:

$ cd {current_path}/schedule-demo/

then push the project schedule-demo to digdag server using the command:

$ digdag push schedule-demo

(Note: On local Ubuntu setup, to start digdag server execute command ‘digdag server’ on the terminal)

If you are outside the project directory and want to push project use command:

$ digdag push schedule-demo --project /{path}/schedule-demo  --revision schedule-demo-v1

{path} => path to the schedule-demo project

Then, launch a browser (any) and add URL ‘http://localhost:65432/’ and under Workflow tab you will see the result similar to below image (you will have to wait for at least 5–6 minutes to get similar output as of mine):

Alt Text

If you observe the image, the output with ID 2 and ID 5 has 2 minutes difference (refer column 'Last Attempt' ) and the value of Workflow is two-minute. You will find similar result for the one-minute workflow (i.e. the execution time difference between every present and next workflow attempt will be 1 minute).

P.S. There could be different ways to achieve the result demonstrated in the above example. This is just one of those :)

If you enjoyed this blog, share it with your friends!

Thank-You for reading!

Sources:

https://www.digdag.io/
https://docs.digdag.io/getting_started.html
https://docs.digdag.io/workflow_definition.html?highlight=include#include-another-file

Retry later

Top comments (2)

Collapse
 
hiroysato profile image
hiroyuki sato

Maybe, you can use cron> syntax

schedule:
  cron>: "0 4,12 * * *"
Enter fullscreen mode Exit fullscreen mode

Scheduling workflow

Collapse
 
shubham_kadam profile image
Shubham Kadam • Edited

Yeah, that works too.

Image of Docusign

🛠️ Bring your solution into Docusign. Reach over 1.6M customers.

Docusign is now extensible. Overcome challenges with disconnected products and inaccessible data by bringing your solutions into Docusign and publishing to 1.6M customers in the App Center.

Learn more