Is Jenkins Pipelining Really This Messed Up??

I've been banging my head against Jenkins pipelines over the last few days - the more I try to work to set up a simple example of re-usable code, the more I feel that, in this respect, Jenkins Pipelines and Groovy are conspiring to prevent you from practicing clean coding and proper IaC.

Base Note

Note that if you create a pipeline job in Jenkins and point it at a main.groovy file, it will run it in the Groovy interpeter. So the following:

// File ./main.groovy
println "Hello world!"

... is a valid file for the Jenkins pipeline definition. It will just print its message and exit as a successful build.

For all intents and purposes, the requirements on the file is just to be a Groovy script.

Then comes the fun part.

Imports

In any sane programming language, importing a file in the same folder as your script should be easy and obvious , especially if the language purports to be a scripting language.

In plain Groovy, this is actually not too hard - you just need to remember that it is kin with Java, and some prerequisites need to be taken care of. The following works fine:

// File ./main.groovy
import zoo.Cat

kitty = Cat()
kitty.sound()

// File ./zoo/Animal.groovy
package zoo
def sound() { println this.my_sound ; }

// File ./zoo/Cat.groovy
package zoo
import zoo.Animal

class Cat extends Animal { // Animal is implicitly a class!
  def my_sound = "meow"
}

If you run groovy main.groovy the import is successful.

Try running that in a pipeline job, and it will fail, complaining that it cannot resolve import zoo.Cat. The reason for this, of course, is because the CLASSPATH is that of the parent environment - which has no fore-knowledge of our dynamically (from the running process's point of view) loaded Groovy script.

So that avenue is fscked.

Parse. Evaluate. Fail.

For the simple question "how to import another groovy file" there are myriad, insane, suggestions on StackOverflow, most of which I chose to ignore because they're patently bat ship crazy (or the platform is pathologically senseless). The practice is necessary however, to get around the classpath issue.

There are two that seem reasonable enough. The first I came across was this:

GroovyShell shell = new GroovyShell()
def script = shell.parse(new File('zoo/Cat.groovy'))
script.method()

I am sad to report that this does not work in Jenkins, because it nerfs the use of GroovyShell. On the one hand, this is probably a security feature to prevent running arbitrary code on your Jenkins instances ; on the other hand, what IS our code if not arbitrary from Jenkin's point of view? And to be perfectly honest, I could very easily just run curl $url | bash as a step and bypass Jenkins's security restrictions altogether. So this definitely feels like a nerfing.

On my setup, I do get a message by which the restriction can be "approved" by administrators (the rights of which I do have) but to no avail.

A similar situation arises with the other tentative solution which I would have hoped to be the "standard" way of doing things:

evaluate(new File('zoo/Cat.groovy'))

Needless to say, these are failed routes as well, and I'm really starting to get cheesed off, and that's rude to the cheese.

Load? Load where?

In the Jenkins Groovy environment, there is a load() function that allows you to do a similar thing to importing. This can work:

// File ./main.groovy
node('') {
  stage("Load a file") {
    kitty = load("zoo/Cat.groovy")
    kitty.meow()
  }
}

// File ./zoo/Cat.groovy
def meow() {
  println "Miaow."
}

return this

Note the return this at the end - this auto-instantiates an object of an implicit Cat class (name taken from the file's name).

This, however, does not work:

// File ./main.groovy
kitty = load("zoo/Cat.groovy")
kitty.meow()

Instead, it fails with an error:

org.jenkinsci.plugins.workflow.steps.MissingContextVariableException: Required context class hudson.FilePath is missing

which is Jenkins whining at us that, unless it's encapsulated inside of a Jenkins node, importing a file cannot be done. Which is pants, because Jenkins itself already peeping well did a repo checkout to get this pipeline file in the first place!

We need therefore to use a "pipeline controller" node (for lack of a better name) which, incidentally, cannot be a node that you intend to use for actually processing builds (unless you want to risk your pipeline waiting on a queued build, which is waiting for.... your pipeline control job to stop running, which is waiting on ...)

Which means we actually need to do this:

// File ./main.groovy
node('controllers') {
  stage("Load a file") {
    checkout([
        $class: 'GitSCM',
        branches: [[name: "master"]],
        userRemoteConfigs: [[
            credentialsId: 'github-pat',
            url: "https://github.com/org/reponame"
        ]]
    ])

    kitty = load("zoo/Cat.groovy")
    kitty.meow()
  }
}

(the prior example I said "can work" only does when re-using a workspace)

Incidentally, that unweildy checkout block cannot be wrapped in an external function in its own file - because we need to check out the repo on the node before we can access the file! Yes there is a shorter notation for simple use-cases, but when you have to take into account custom settings... if you find yourself re-using this frequently it is repeated code in every pipeline definition. BAD.

But it is what it is. Using load() and resigning myself to needing a pipeline controller node, I can finally get my file separation. I can even do this:

// File ./zoo/Cat.groovy
def meow() {
  node("farm") { // Run somewhere other than the controller node
    stage("Sound the farm") {
      println "Miaow."
    }
  }
}

return this

... which effectively allows me to dynamically add stages as I go along.

But you can Share your Libraries!

Listen, the Shared Library concept they're peddling sounds like a good idea on the surface, but do you really expect me to farm out a subset of files another repo, go through the Jenkins GUI to add it in with a custom name, the linke between the two being in the platofrm instead of in the code when the files are meant to be right there next to eachother like in any sane development project??

Declarative Pipelines? Don't declare victory

All this is good and well, but what if we want to be a bit cleaner and not use imperative programming, but instead use the actual Declarative implementation that Jenkins is really wants us to use?

Well you're stuffed.

You can only run script code inside script{} blocks, which can only exist inside stage{} blocks in stages{} blocks in a pipeline{} block. So farming out the pipeline stages is not possible at all, you can only isolate the actual build script stuff, by which point you're writing in shell or Makefile or whatever anyway so why bother ducking around with Groovy.

Note that the parameter {} declaration is a no-op when not used in a declarative pipeline as well, so that's all back to the GUI unless you like repeating yourself.

---

And at this point, I give up.

• There should be a native DSL expression for pipelines that allow stashing a subsection of pipelines in a separate file
• WITHOUT having to create ANOTHER repo, and adding it MANUALLY (what the flap is IaC for anyway?) in the administrative interface
• And TBH it should NOT need to tie up another instance, thereby requiring extra hardware because a language is half-baked.

So I'm stuck with Jenkins and a homebrew import solution without declarative goodness, just so I can have clean code because Jenkins doesn't seem to realise that I don't actually WANT to maintain several copies of my code, NOR fork my pipelines out to an extra repo, NOR use its administrative GUI to load libraries. Oh, and you still set up jobs via obscene amounts of GUI configuration. In pursuit of IaC.

---

Thank Chuck it's the holidays and I can step away from this for a couple of weeks.

Comments

[Raphael Habereder]

I know your pain. Having spent a surreal amount of time with Hudson/Jenkins over the last 12 years, there are countless things that bug me.

Jenkins pipeline groovy DSL uses a custom CPS interpreter with a few limitations during the interpretation of groovy. This means that it bypasses/overrides a lot of the standard implementation of groovy. It never felt like "real groovy" to me. I can't recount the times I opened my IDE and just coded groovy code, only to be frustrated to see it fail in Jenkins.

If you want to get closer to "real groovy", you'll have to work a lot with the @NonCPS Annotation, which itself brings a whole slew of different problems again if you mix your NonCPS Methods with regular CPS Steps.

Or the infamous parallel index-loop bug, which was declared as "feature", where when using a parallel step for an indexed loop, the index would always be at the max value, completely breaking your loop. There were many painful bugsFeatures like this that I encountered in my career.

What I have learned is, "don't try too complicated things and Jenkins will be good/ok to use". The moment you want to do anything that defies the typical "Jenkins CPS Step Glory", you are going to have a bad time.
Define your own steps, create a shared library and pray that it "just works". But don't overdo it with the library, or else you'll easily find yourself in maintenance hell.

I do love Jenkins, it has served me well in many a project, but nowadays? It would strongly depend on what I have to do. If I can use Concourse, ArgoCD, Drone or Tekton, I'll be super happy. Jenkins on the other hand has gone way down on my "happyness list" in the last years.

[Matt Morgan]

Great points. The horribleness of the shared library implementation in Jenkins was the real tableflip for me. I eventually had to arrive at the only reasonable solution: don't use Jenkins.

My company switched to codefresh.io with no regrets. Would also very much consider github actions or AWS codepipeline or literally anything else (I hear gitlab has good stuff and there's circle and travis, etc.) before ever giving Jenkins another look.

[Zevrant]

It's quite apparent that the author as well as the commenters here have a very limited understanding of how to actually use Jenkins that it moderately infuriating. Having been working with jenkins since college (more than a decade), if you know what you're doing it's quite easy to write and maintain a proper library for Jenkins to use complete with unit tests. Loading random groovy scripts is 'nerfed' because this is a security issue more than anything, I shouldn't be able to bypass security constraints by running random groovy scripts wherever.

You don't need to 'farm out' the library code into a separate repo. In both my home lab and several shops i've worked in over the years use a single git repository for all of the jenkins code. While yes multiple sparse checkouts are required, that's a small amount of overhead for having a centrally managed system in place for maintaining all of your jenkins code.

There should be a native DSL expression for pipelines that allow stashing a subsection of pipelines in a separate file
there is it's done by using library code, or calling a downstream job. Use the features that exists instead of bashing something you clearly don't know how to use

WITHOUT having to create ANOTHER repo, and adding it MANUALLY (what the flap is IaC for anyway?) in the administrative interface
See above comment

And TBH it should NOT need to tie up another instance, thereby requiring extra hardware because a language is half-baked.
Nope. try again. If you're running on kubernetes as most large scale jenkins applications tend to do these days, you spin up a set up reusable containers in a pod and reuse them. If on bare metal just reuse executors on your nodes, this incurs very little overhead.

Learn how to use the tools before you bash them. Anyone who actually knows how to use jenkins can easily poke holes in all of your arguments