DEV Community

Cover image for A serverless solution to just-in-time scheduling
Adèle Gauvrit for Kumo

Posted on

A serverless solution to just-in-time scheduling

TL; DR: We published a CDK construct to schedule events precisely.
Documentation is available on the project's github

When I worked on a serverless CMS with Guillaume, we had to find a way to schedule an event (publishing a new article)
at a precise time with serverless tools. AWS doesn't offer a built-in service to do so. We found straight-forward options but none as precise as we wished. We came up with a solution of our own. We published a CDK construct for you to add our solution to your stack without boilerplate.

Naive approaches to serverless scheduling

Scheduling a lambda to run every minute

A first approach is to use lambda schedule trigger feature. Lambdas can be triggered every down to every minute.
Every minute, you can query your database to find if any events need to be published now. If so, run the action you planned (like publishing an article), else do nothing.

Trigger a lambda every minute to check if any events needs to be scheduled

A lambda can be triggered down to every minute and the trigger time has a one-minute precision guarantee. It's good enough for some use-cases but not if you are looking for a highly precise method.

Using DynamoDB time-to-live and streams

DynamoDB comes with a time-to-live feature. You can add a ttl attribute to an entry. At said time, AWS will remove the entry form your table. If you have a stream plugged to your table you can play an action at scheduled time.

However, this approach is even less precise than the first. AWS only guarantees a forty-eight-hour precision on the time-to-live date. In practice, users observe a ten-minute delay but it's not guaranteed. If, like us, you work on time-sensitive events, you will want to avoid this method.

Use Dynaomdb time-to-live feature to trigger events with one-day precision

Introducing cdk-scheduler

To match our just-in-time ambition, we leveraged the SQS delay feature. When publishing on an SQS queue you can set a delay that goes up to fifteen minutes.

cdk-scheduler leverages SQS delay feature to trigger events precisely

Instead of starting a lambda every minute, we do it every fifteen minutes, which reduces fixed costs. This method has a one-second precision gap!

In case you're in hurry to schedule, we also added a "near-future" handler: a second lambda is plugged on a DynamoDB stream. It handles events that are to be scheduled in less than fifteen minutes after their creation and pushes them directly on the SQS with a delay. This edge case is just as precise.

Pricing

CDK-scheduler lambda calls are low. Fixed costs for lambda execution is $0.06/month. Regarding variable costs, CDK-scheduler costs $2.50 for one million scheduled events
($1.25 to put your items in DynamoDB and $1.25 to delete them, SQS is free for the first million events).

Comparison

Solution Fixed costs (per month) Variable costs (one million events) Precision
CDK-schedluer $0.05 $2.50 1 second
Scheduled lambda $0.80 $2.50 2 minutes
DynamoDB Time to leave $0 $2.50 1 day

What's next?

Our construct is now published. Documentation is available on our open-source repository. If you would like to try it on your project run yarn add cdk-scheduler. It fits the need we had when we developed a CMS like a glove. We're now looking for your feedback. What do you need when scheduling an event? What configuration or features would you like to see?


Cover photo by alexandru vicol on Unsplash

Discussion (5)

Collapse
m0ltzart profile image
Roman • Edited on

You can use a step function with Wait step. Which can accept both relative seconds and absolute ISO timestamps.

docs.aws.amazon.com/step-functions...

Collapse
guiyome profile image
Guillaume Égée

Step functions are probably the best way to schedule an isolated task.
However, to manage lots of scheduled events, you would create multiple step functions that are difficult to manage (unless you use a database to index them) and impossible to make evolve once created.
Using this CDK scheduler, the scheduling feature is managed by an isolated micro-service which is only responsible of its function: scheduling.

Collapse
adelegauvrit profile image
Adèle Gauvrit Author

Thanks for the feedback Roman, I'll look into this solution as well for scheduling

Collapse
seriousme profile image
Hans Klunder

What about creating a scheduled rule on EventBridge ?
Its only accurate up to a minute, so less exact than the Stepfunctions suggested ;-)

docs.aws.amazon.com/eventbridge/la...

Collapse
guiyome profile image
Guillaume Égée

Yes, cron or rate expressions are useful for recurring tasks, and are the simplest way to schedule tasks with a minute precision. In this use case, I agree that an EventBridge scheduled rule would be the best option!
However, this CDK constructs aims to address another use case : scheduling non-recurring events and be able to manage them. Thanks to the DynamoDB database, you create, delete, count scheduled events, a feature that you won't have with EventBridge nor StepFunctions.