Recently, as part of work on version 3 of Boomerang Flow, we undertook a decision to migrate from Kubernetes Jobs to Tekton Tasks and TaskRuns.
As we stepped back and looked at the how and after many deep conversations late at night (thanks Marcus), we decided that it was also the right time to migrate from the existing Kubernetes Java Client to the Fabric8 Kubernetes Java Client.
What is Boomerang Flow
Boomerang Flow is an open-source event-driven cloud-native workflow automation tool that runs on Kubernetes and provides a no-code drag and drop editor for workflow creation. We focus first on the end user experience and then the technical features.
Why Tekton
It was a big decision to make, one that had been in the works for a while with over 9 months of healthy discussions, and so at the start of 2021 we commenced the changes.
Whilst we had done a lot of great work in our internal controller to orchestrate and manage Kubernetes Jobs, there were a number of areas that we lacked features or had alternative ways of achieving the same result, but without the broader community. We were also starting to see cracks in our error handling
We also wanted to make sure that by adopting Tekton we would not be sacrificing any end-user experience. In the following comparisons you'll see that in majority of the cases we could actually build out new functionality to help our users.
For now, we are only adopting the Tekton Task/TaskRun and working in with our existing Workflow DAG, it opens up a huge range of benefits;
- Leverages the community of a popular open-source system
- Provides additional Task functionality with steps and script blocks
- Provides a wider range of Tasks built by community
- Allows us to focus less on adding new functionality to the engine, and spending this time on enhancements with the community.
Tasks
We had Tasks, Tekton has Tasks. The models were relatively the same. Fantastic! Alignment would not be a pain.
We defined our Tasks in JSON, Tekton Tasks are in YAML. The great JSON vs YAML debate took place and we settled on YAML as this is only for the advanced user and does not take away from the end-user experience. Plus, everything is YAML!
We did however have metadata and elements needed for the UI and as such we have followed the Kubernetes approach and stored this metadata in the annotations and default it when importing a straight Tekton Task YAML without the optional annotations.
We did not support multi-step or script blocks so it was a win-win to be able to bring this to our users. Whilst the typical user does not see any of this, they drag and drop a task from a Palette onto the Workflow Editor and provide inputs, it allows us to provide more powerful and complex tasks and task management, abstracted by a powerful experience.
Outcomes
- New YAML editor as part of Task Management
- Experience metadata stored in the annotations
- Additional Task features with script, working directory, environment variables, result parameters, and multi-step (coming soon).
Parameters
We had Properties, Tekton has Parameters. We had ${p:property} as the syntax, Tekton has $(params.parameter). We passed in all properties to a task, with Tekton you have to explicitly define the parameters wanted.
The biggest difference was in how we approach the layering and substitution of Parameters. Boomerang Flow offers additional layers for the user to have greater control as to how to store and re-use information.
Boomerang Flow also provides powerful inheritance and full recursive substitution, this means that if you define a Parameter at the Team layer and then also at the Workflow level, the Workflow Parameter will win out, this is all resolved as part of using $(params.<parameter>). Or if you have a Workflow Parameter that references another Parameter this will also be resolved and substituted.
| Boomerang Flow | Tekton |
|---|---|
| System | Content |
| Global | - |
| Team | - |
| Workflows | Pipelines |
| Tasks | Tasks |
Outcomes
- Retained the same great experience and powerful inheritance and recursive substitution with all existing layers kept.
- Adjusted to the Tetkon
$( )syntax - Explicitly define the parameters that are sent through to a task.
Result Parameters
We had Output Properties, Tekton has Result Parameters.
We utilized a watcher side car to scrape a folder and turn all files into Output Properties by posting these back using a HTTP Post and then patch a ConfigMap.
Tekton on the other hand requires explicitly defining the result parameters and
- scrapes a folder for files named that match the explicitly defined expected results
- Stores them in the container termination message. This is thus limited to 4096 bytes
- relies on alternate resources for larger outputs.
What we lose in this instance we gain in other areas, including in not having a custom implementation that required patching ConfigMaps that sometimes failed. We will also be submitting a TEP to the community on hopefully adjusting how parameters are stored to remove the size restriction.
We also were able to update the UI to now provide auto complete when reference the result parameter of another task. Isn't that neat!
Outcomes
- Explicitly defined Result Parameters in Task Manager
- Deprecation of our bespoke Watcher and related code
- A more stable well tested implementation
- Auto prompt reference of result parameters when in the Workflow Editor
- New description when viewing results in the Workflow Activity.
Error handling
Another big win was solving a number of our error handling concerns. Tekton provided a structured Task error object and error handling, and watching for these errors was made a lot simpler with the Fabric8 client.
We could then capture and translate these to nice user based messages to return to the user and provide greater visibility and detail at both the Task and Workflow level. This was a powerful enhancement for our user experience.
Outcomes
- More structured and detailed errors
- Better ability to watch and handle the errors
- New Task and Workflow error viewing in the Workflow Activity for greater visibility.
Why change Java clients
Technically we did not have to move from the vanilla Kubernetes Java Client to the Fabric8 Kubernetes Java client. We could have achieved the required CRDs (Custom Resource Definitions) in the existing client... that being said, it was much easier with the Fabric8 Tekton extension.
The Fabric8 community itself has also been very helpful and its these experiences that help provide the level of examples and references for advanced implementations such as a Custom Watcher for TaskRun execution.
Acknowledgments
This migration would not have been possible without the blogs and information from the community, in particular
- Difference between Fabric8 and Official Kubernetes Java Client
- Access Tekton Pipelines in Java using Fabric8 Tekton Client
- Fabric8 Kubernetes Client Cheat Sheet
- Code from the Che project
And my fellow contributors: Ben, Marcus, and Costel.
Show your support
Do not hesitate to support us if this helped you:
- ⭐️ the GitHub repository
- 💬 Follow us on Twitter
- Install the project on your own Kubernetes cluster
- 🙏 Join us and help contribute
Top comments (0)