A crash course in classpaths: Build

The cover image represents Gradle's class loader hierarchy.

Welcome to part 2 in my series on classpaths and class loading in Gradle projects. If you haven’t read part 1, I would recommend checking it out first, as it lays the foundation for what is to follow. In this part, we will be focusing on the classpaths available to the build process itself.

What do we mean by “the build”?

As I alluded to in the first post, modern JVM development (nor least Android development) rarely if ever involves direct interactions with SDK tools like javac or kotlinc. Instead we “run builds” with a “build tool.” Under the hood, these build tools (such as Gradle), invoke the more fundamental SDK tools. As an example, Gradle's java plugin registers a task, compileJava, of type JavaCompile; during its execution, that task will invoke the Java compiler, javac.

Now we can give a simplified definition for "the build":

The build is a high-level abstraction over the more fundamental SDK tool operations that compile and run your software applications.

How does this relate to classpaths and class loading? That's up next.

The build classpath(s)

Build scripts in a Gradle build must be compiled, and then they must be run, in order to execute your build. This process is automated by Gradle, regardless of whether your script is written in Groovy (.gradle) or Kotlin (.gradle.kts). When you execute a build (whether that's clicking the green arrow in Android Studio or running ./gradlew app:assembleDebug from the command line), Gradle will execute that instruction in two distinct steps: first it compiles your build scripts, and then it runs your build. If you're familiar with the Gradle build lifecycle, it may help to think of the first step as being in the configuration phase, while the second step is in the execution phase.

As with any other compilation in the JVM, compiling a script requires a compile classpath, which we call the build compile classpath. Let’s see what it looks like through a simple example. To do so, we will be using Gradle build scans as a convenient visualization tool.

A simple example to set the stage

Let's start by considering a very simple build script and inspecting the build classpath, which is on the Build Dependencies section in a build scan.

In the examples below, I am deliberately using gradle and not ./gradlew. I'm doing this to demonstrate that you can use the global installation of Gradle on your system to build projects, and to simplify the example setup. Just create a build.gradle and run it!

// root build.gradle
println "project name = $name"

We can then execute gradle help --scan, and we will see the following amidst the rest of the output:

> Configure project :
project name = classpaths-example

We'll also see a build scan for this simple build. These are the build dependencies:

As you can see, there are none. While this may not seem surprising, given the simplicity of the script, it's not strictly accurate. First of all, we can infer from the fact that our println statement worked that the Groovy Development Kit (GDK) is on the classpath, and furthermore that so is the JDK. (println is Groovy shorthand for System.out.println().) Finally, the name variable is a property on the Project instance that is backing our Gradle script, which means the Gradle API is also available by default. So, our build scan notwithstanding, Gradle provides quite a lot to us on the build classpath, by default.

Finally, there is one more dependency that is hidden. Our usage of the --scan option implicitly adds the Gradle Enterprise (GE) plugin to our build. It's unclear why that isn't visible in the scan. If you add an empty settings.gradle and run that build again, you'll get a new scan:

And now you can see that the GE plugin is a build dependency.

Adding a plugin to your project

But what's a Gradle project without a plugin? Let's add a plugin and see how it changes the build classpath.

// root build.gradle
buildscript {
  repositories {
    mavenCentral()
  }
  dependencies {
    classpath 'org.jetbrains.kotlin:kotlin-gradle-plugin:1.4.21'
  }
}

(Yes, mavenCentral)¹

If we configure this project with gradle help --scan (as we like to do), we can see what this has done to our build classpath:

There's our Kotlin Gradle Plugin (with transitive dependencies) on the build classpath of the root build script. You'll note we haven't even applied the plugin at this point, let alone configured it in any way. Rather, we have explicitly added it to our build classpath using the venerable buildscript block.

While I'm certain most, if not all, of us have seen code like that, we have to admit it's a little verbose. Why should we have to explicitly specify our plugin dependencies, especially when there often isn't an obvious mapping from the dependency coordinates to the plugin ID? (Simply kotlin, as in apply plugin: 'kotlin', in this case.) There's something suspiciously cargo-cultish about it.

The plugins block

The alternative to explicitly adding to your build classpath is to use the plugins block. Let's convert the above example and see what, if anything, changes.

// root build.gradle
plugins {
  id 'org.jetbrains.kotlin.jvm' version '1.4.21'
}

This block combines adding Kotlin Gradle Plugin (KGP) with applying it, so its behavior is a little different from the above.

Sometimes it can be useful to split these operations and maintain an explicit buildscript block, but the scenarios in which that makes sense are rapidly disappearing. They're also outside of the scope of this post!

If we once again run gradle help --scan, we can inspect our build classpath:

As we can see, it's very similar to the other example. There are three important differences:

1. We can see the additional level of nesting, with a top-level dependency of org.jetbrains.kotlin.jvm:org.jetbrains.kotlin.jvm.gradle.plugin:1.4.21. This is known as a plugin marker artifact, and is how Gradle manages the problem of mapping a plugin ID to its "physical" artifact (a jar and its transitive dependencies).
2. If you inspect the build dependencies in the two scans, you'll see that, in the first, they were all resolved from https://repo.maven.apache.org/maven2/ (aka mavenCentral()), whereas in the second they were all resolved from https://plugins.gradle.org/m2 (aka gradlePluginPortal()). There are ways to change the repository(ies) from which plugins are resolved, and we'll discuss them in just a bit.
3. As we already noted, plugins also applies the plugin. You can verify this by looking at the Plugins link on each build scan.

Actually, you can use plugins without applying a plugin

I know I know, I kind of made a big deal about how using plugins automatically applies the plugin to your build. While that is the default behavior, you can tell Gradle not to do it like so:

// root build.gradle
plugins {
  id 'org.jetbrains.kotlin.jvm' version '1.4.21' apply false
}

There are two scenarios I know of that makes this a useful pattern. In the first, you just need the Kotlin plugin on your build classpath for some reason, maybe because another plugin expects it (for reasons we will touch on in the next post). In the second, this is a way to force all subprojects in your build to use the same version of the given plugin. Consider this:

// settings.gradle
include ':app'

// root build.gradle
plugins {
  id 'org.jetbrains.kotlin.jvm' version '1.4.21' apply false
}

// app/build.gradle
plugins {
  id 'org.jetbrains.kotlin.jvm'
}

You know what's next: gradle help --scan (link).

We can see from this that the build classpath is fundamentally unchanged (although we can now see the Gradle Enterprise plugin, since we added a settings.gradle). This confirms that KGP has not been added to the build classpath of the app subproject: it is only on the classpath of the root build script. Nevertheless, subprojects can access it thanks to Gradle's well-defined (though poorly documented) class loader hierarchy that makes the root class loader the parent of all subproject class loaders. These subproject class loaders delegate to their parent in the normal way, as discussed in the first post of this series.

Android Gradle Plugin

In contrast to KGP, most versions of AGP cannot be applied simply with plugins, because it only started publishing plugin marker artifacts with the 4.2.0 series. Nevertheless, using the modern plugins approach with AGP 4.2.0-beta04 provides a good opportunity to show off an important feature of Gradle that I don't think many people know about.

// settings.gradle
pluginManagement {
  repositories {
    google()             // AGP
    gradlePluginPortal() // KGP
  }
}
include ':app'

// app/build.gradle
plugins {
  id 'com.android.application' version '4.2.0-beta04'
  id 'org.jetbrains.kotlin.android' version '1.4.21'
}

There are two interesting features of this example: we no longer need a root build.gradle, and we have this new pluginManagement block in our settings script. The first follows from what we've learned so far, and the second is a bit out of scope, so I encourage you to read the docs. It is quite powerful and can do more than just specify repositories. Settings scripts — not just for include statements anymore!

Gradle build script compilation

With this foundation, we can return to the beginning with new eyes. What is the build? What is its relation to classpaths? In this post we have, fundamentally, been talking about the build compile-time. Our goal may be to compile our projects, but before we can do that we have to compile our build scripts. Everything we've learned in this post boils down to different ways to influence the build classpath so that we can compile our build scripts and then, ultimately, compile our projects. It really is classpaths all the way down.

What's next?

Now that we have a pretty good handle on the "build compile-time", we can next ... well, learn one more trick for influencing the compile-time, and then focus on the "build runtime." For the former, we'll learn about a useful and, frankly, cool monkey-patching technique, while for the latter we'll focus on considerations for plugin authors.

See you then!

Special thanks

Thanks once again to César Puerta for thoroughly reviewing this post and helping me keep my foot out of my mouth at several key points.

Endnotes

¹ Pour one out for jcenter. up

Comments

[Matan S]

Regarding pluginManagement. If our organization uses a custom artifact repository (such as Artifactory). Will defining Artifactory in pluginManagement enables using "plugins{}" block to apply our custom plugin?