DEV Community

Amara Graham
Amara Graham

Posted on

My First Flow with Kestra.io

#beginners #automation #showdev #tooling

As I'm learning more about Kestra.io, the community, and the possible use cases, I wanted to build a small PoC (proof of concept) that resonated with me and leveraged a fair amount of the platform aspects so I could get some hands on learning beyond the existing tutorials and examples (which are great! Give them a try!).

After talking with a coworker, I loved his daily reminder to log off scheduled on his calendar. This kind of daily ritual is really great for wrapping up your day and leaving work at work. Or at least trying to.

I told him I was going to 100% steal his idea.

Let me walk you through what I'm doing and learning as I go. This won't be a tutorial, but you can see how I learn and explore a new platform, following along with links back to the docs if you want to learn even more.

Requirements of the PoC

I wanted to make sure this PoC covered multiple aspects of the platform, even if it might be a little trivial at times. I wanted to see the dashboards populate with some data, use a few plugins, and start to see the difference between the OSS Edition you can access for free and the paid experience with the Kestra Enterprise Edition.

I needed to interact with the following during my daily or weekly shutdown:

  • Scheduler/chron - what? You think I would remember to do this daily?
  • Slack - our main communication tool and where my weekly recap ultimately needed to land, but also where I'll get the reminder
  • GitHub - no surprise here, where the bugs, feature requests, issues, and PRs live for the source code and docs
  • Notion - Throw this recap into my "Weekly recaps" so it doesn't get lost on Slack
  • Google Calendar - Read my calendar and remind me who I talked to today or this week

All of these looked like plugins that should reasonably work well for my PoC. Excellent!

For my first iteration, I won't make this too terribly smart and it will prompt and grab the same info everyday, just for that day. Maybe v2 will do something more comprehensive on Friday when I actually write my weekly recap.

After a quick search through the Blueprints, I didn't find anything specific that I wanted to build off of, so I ended up starting from scratch. I usually recommend building from or extending tutorials and existing examples.

I should also mention I'm building this flow with the flow code editor, directly editing the YAML.

What's a plugin?

Plugins are the essential building blocks for tasks and triggers in your workflow in Kestra. This might be a subtle shift from some platforms where the plugins are extensions or integrations, which they are in Kestra too, but they also include core functionality. You'll probably end up using several plugins in your flow, and many more if you are doing something non-trivial.

See if you can spot the plugins I use in the YAML snippets below. You'll see a mix of 3rd party integrations and core functionality.

Kestra UI

Learning new tools always requires spending a little time mapping your existing knowledge to the UI and terms.

One of the things that I really struggled with early on was the purple, primary action button in the top right corner of the UI changes depending on what screen I'm on. So it could be used to create a flow, execute a flow, create a user (in Kestra Enterprise Edition), etc.

Leave me a comment and let me know how you feel about this UI design choice. Is it intuitive for you? Do you want a more descriptive button name?

Triggers

You probably want to do something when something happens, whether that's a moment in time or based on an event like a form being submitted or new data arriving. You need a trigger for that. For my PoC, I want this to run every weekday around 4pm, when I'm thinking about wrapping up for the day.

triggers:
  - id: daily_shutdown
    type: io.kestra.plugin.core.trigger.Schedule
    #UTC Austin @ 4pm (could add timezone), except weekends
    cron: 0 22 * * 1-5
Enter fullscreen mode Exit fullscreen mode

I like these at the top of the YAML because they show me exactly when this flow happens.

Tasks

Flows require at least one task. You simply cannot save a flow without a task. This makes sense because why would you have a flow that does nothing.

I'll have several tasks in my flow, but I want to make sure I see a reminder or prompt in Slack. That will look something like this with the Slack plugin:

id: send_reminder_message_to_slack
    type: io.kestra.plugin.notifications.slack.SlackIncomingWebhook
    url: "{{ secret('SLACK_WEBHOOK') }}"
    payload: |
      {
        "text": "Time to shutdown"
      }
Enter fullscreen mode Exit fullscreen mode

I'll need to revisit this to add something more interesting to the payload (like info from other tasks), but sending a Slack message is as easy as getting the tokens and secrets configured.

Eventually, it will look like this:

id: send_meetings_to_slack
      type: io.kestra.plugin.notifications.slack.SlackIncomingWebhook
      url: "{{ secret('SLACK_WEBHOOK') }}"
      payload: |
        {
          "text": "Here are the meetings you went to today: {{ outputs.get_calendar_events.events }} "
        }
Enter fullscreen mode Exit fullscreen mode

But we'll talk more about outputs a bit later.

Logging

I ended up doing a lot of logging tasks between the tasks that used plugins, mostly to verify what was coming back in the response and if I wanted to do anything else.

- id: get_cal_list_log
    type: io.kestra.plugin.core.log.Log
    message: "{{ outputs.get_calendar_events }}"
Enter fullscreen mode Exit fullscreen mode

I'm still having some trouble getting the Google Calendar plugin to send more than an empty array back, so for now I'm logging the whole object I get back to see if I get any clues.

Flowable tasks

I threw in a flowable task because depending on how you classify a meeting, I don't have meetings everyday. I do have a blocker for lunch everyday, which is technically a meeting...

- id: if
    type: io.kestra.plugin.core.flow.If
    condition: " {{ outputs.get_calendar_events.events  == [] }} "
    then:
    - id: send_no_meetings_to_slack
      type: io.kestra.plugin.notifications.slack.SlackIncomingWebhook
      url: "{{ secret('SLACK_WEBHOOK') }}"
      payload: |
        {
          "text": "Hooray! No meetings today."
        }
    else:
    - id: send_meetings_to_slack
      type: io.kestra.plugin.notifications.slack.SlackIncomingWebhook
      url: "{{ secret('SLACK_WEBHOOK') }}"
      payload: |
        {
          "text": "Here are the meetings you went to today: {{ outputs.get_calendar_events.events }} "
        }
Enter fullscreen mode Exit fullscreen mode

I really like the human readable way this, well, reads. If, then, else shows me what the condition is and what's going to happen for the two branches.

I know some platforms allow you to define a default path for flows, but I didn't see that here, which I actually really like. Introducing a default path feels confusing when you should rely on the evaluation of the condition.

As I mentioned above this is currently always getting [] back from the Google Calendar plugin, so apparently I never have meetings, but when it works I'll have that taken care of with this flowable task.

Outputs

I added an output that was trivial because I was most likely going to send the summary of the work I did to Slack in the message payload and wasn't sure if there would be any difference if I did or didn't have an output.

outputs:
  - id: sent_bool
    type: BOOLEAN
    value: "true"
Enter fullscreen mode Exit fullscreen mode

In the execution outputs, I wanted to see something I added and where it would show up in the UI. Again, trivial, because in this iteration I don't anticipate this flow making data available for a subsequent flow.

I did use outputs across the various tasks too, so I could use the data from one task in the following task.

Revisions

Don't do what I did as a lazy developer and put my secrets right into my flow. They stay in your revision history. Plain text. Oops.

Secret handling in the Enterprise Edition is too easy to not use, so don't put your secrets plain text in your flow. Keep your secrets secret. Then use an expression like below:

{{ secret('SLACK_WEBHOOK') }}
Enter fullscreen mode Exit fullscreen mode

Revisions become really powerful as you are making changes to your flow, particularly while you are building and testing because sometimes a change doesn't work like you want, and through revisions you can see exactly what changes were made.

Where I'm at

I love that I can export an image of my topology directly in Kestra.

So for now I don't have a "working" increment that I'm happy with. While it is firing every day, I still need to dig into why Google Calendar thinks I have no meetings... when I definitely have meetings. I'm currently playing around with public/private event visibility to see if that changes anything. More on that next week I'm sure when I also add the Notion and GitHub plugins.

Have you built something with Kestra? Link me to your repo in a comment below.

Top comments (1)

Collapse
 
deltax profile image
deltax •

Strong write-up.
I like the framing: cron as a trigger, not a decision layer. Treating scheduling as a signal and keeping all validation/context in the application avoids false assumptions (time, idempotency, duplication, DST).
Same pattern scales well to workflows, automation, and AI outputs: wake → measure → decide elsewhere.
Clean separation, fewer hidden bugs.