Gradle Under the Hood: ClassLoader Harvesting vs. Network CI Crashes

In the first article of our Gradle Under the Hood series, we explored how we decoupled incremental build tracking from volatile compiler outputs to achieve bulletproof caching.

Today, we dive into a critical continuous integration (CI) dilemma that almost every plugin author faces: How do you execute complex, isolated task workloads on headless CI nodes without crashing due to missing network dependencies or restricted local caches?

Let’s unpack how we engineered the JPersist Suite verification engines to harvest their own running binary classpaths on-the-fly, how that architectural assumptions shattered across older Gradle versions, and how we leveraged native classpath streaming to achieve bulletproof reliability in our latest v1.8.1 patch.

The Headless CI Dilemma

To protect the Gradle build daemon from memory leaks and classpath contamination, advanced automation tasks must run inside a sandbox. Gradle provides the magnificent Worker API, allowing you to execute tasks via isolated classloaders (classLoaderIsolation):

workerExecutor.classLoaderIsolation { workerSpec ->
    workerSpec.classpath.from(toolsetFiles)
}.submit(MyValidationWorker) { params -> ... }

Where it breaks on CI runners:
To populate toolsetFiles, a plugin typically declares an internal detached configuration and injects remote text coordinates (GAV format):

project.dependencies.add(detachedConfigName, "io.github.jpersist.tools:persistence-tools-api:3.0.0")

When running in a standard developer workspace, this works fine. But in a highly isolated, enterprise continuous integration runner (or mono-repository workspaces where individual component segments are built independently), this text-based lookup introduces massive fragility:

1. If the required companion library artifacts have not been officially merged and pushed to Maven Central yet, the resolution crashes with a ModuleNotFoundException.
2. If your CI pipelines restrict or completely disable access to mavenLocal(), sibling projects cannot share local intermediate build caches.
3. Forcing remote dependency resolution loops over internal libraries during a task execution step slows down your CI velocity.

The First Iteration: ClassLoader Harvesting (v1.8.0)

While building JPersist v1.8.0, we asked ourselves: Why are we forcing Gradle to perform a slow, network-bound dependency lookup against Maven Central for library classes that the plugin itself is already running on?

Since your primary plugin (e.g., hibernate-persistence-gradle-plugin) already declares a dependency on the core tools API module to compile, those physical classes are already loaded into memory by the Plugin’s ClassLoader.

Instead of using text coordinates, we introduced ClassLoader Harvesting. The plugin programmatically queried its own running runtime environment to discover the exact physical .jar file path on disk hosting the core contracts, and fed those file handles straight to the isolated Worker via protection domain lookups:

// Our initial v1.8.0 lookups implementation
Class<?> providerClass = Class.forName(markerClassName, false, classLoader)
def codeSourceLocation = providerClass.protectionDomain.codeSource?.location

This successfully generated a clean local file collection array, completely insulating the task action from network overhead.

The Plot Twist: Cross-Version Version Alignment

We rolled out this design, and our local test suites under modern Gradle 9.x were glowing green. However, real-world integration testing against older long-term support (LTS) streams—specifically Gradle 7.6.6 and Gradle 8.7—unveiled subtle API signature and dependency lifecycle discrepancies that we hadn't anticipated.

1. Lazy vs. Eager Extensions: Under Gradle 9.x, the Configuration.extendsFrom() method natively unwraps lazy NamedDomainObjectProvider targets. In Gradle 8.x, passing a provider directly threw a MissingMethodExecutionFailed exception because the legacy eager API strictly expected a realized Configuration instance.
2. The Transitive Sandbox Leak: In mixed multi-project environments (where one project sub-module runs a legacy Hibernate 5 stack and another runs a modern Hibernate 6 stack), Gradle 7.x's older configuration engine aggressively elevated the dependency graph inside detached file scopes. Hibernate 6 classes accidentally leaked into our Hibernate 5 worker sandbox, stripping away legacy methods and breaking tasks with a critical NoSuchMethodError.

The v1.8.1 Evolution: Native Classpath Streaming

To guarantee 100% universal stability across Gradle 7.6.6, 8.7, and 9.x without hardcoding user-space definitions, we evolved our ClassLoader Harvesting implementation into a more robust two-tier pattern.

First, we isolated the version lookup itself. Instead of scraping runtime class paths on the fly, the plugin now references a lean persistence-plugin-info.properties file generated incrementally at build time. This reads the exact published coordinate version out of a detached configuration, resolving smoothly without leaking transitive scopes:

static ConfigurableFileCollection resolvePersistenceToolsFiles(Project project, Provider<String> jpaVersionProvider) {
    def version = jpaVersionProvider.get()
    boolean isLegacy = isLegacyJpaVersion(version)
    String providerArtifact = isLegacy ? "persistence-tools-legacy" : "persistence-tools-jakarta"

    String toolsVersion = "3.0.0" // Baseline safe fallback
    def props = new Properties()
    def resourceStream = JakartaPersistencePlugin.class.classLoader.getResourceAsStream("persistence-plugin-info.properties")
    if (resourceStream != null) {
        resourceStream.withCloseable { props.load(it) }
        toolsVersion = props.getProperty("persistence-tools.version").trim()
    }

    def detachedConfig = project.configurations.detachedConfiguration(
        project.dependencies.create("io.github.jpersist.tools:providerArtifact:{toolsVersion}")
    )
    return project.files(detachedConfig)
}

Second, we paired this with Native Classpath Streaming. We bypassed raw, unchecked file collections entirely and mapped the task's execution graph straight to the project's native compilation classpath provider.

By feeding sourceSet.compileClasspath directly into the task's configuration container, Gradle's native resolution engine handles conflicts, dependency trees, and caching boundaries naturally at execution time while remaining completely backward compatible:

// Inside HibernateEnhancementPlugin.groovy (v1.8.1)
def enhanceConfigName = sourceSet.name == 'main' ? "hibernateEnhanceClasspath" : "\${sourceSet.name}HibernateEnhanceClasspath"
def enhanceConfigProvider = project.configurations.register(enhanceConfigName) { config ->
    config.canBeConsumed = false
    config.canBeResolved = true

    // Eagerly unpack via .get() for Gradle 8.x signature compatibility
    config.extendsFrom(project.configurations.named(sourceSet.compileClasspathConfigurationName).get())
}

// Wire the lazy file collections straight into the task configuration graph
enhanceTask.enhanceClasspath.from(enhanceConfigProvider)
enhanceTask.enhanceClasspath.from(JakartaPersistencePlugin.resolvePersistenceToolsFiles(project, activeVersionProvider))

The Engineering Win

By combining localized classloader resource harvesting with native compilation classpath streaming, we secured three major structural improvements for our users:

1. 100% Offline Independence: The isolated worker tasks (validatePersistenceSchema, lintNamedQueries, and hibernateEnhance) can run completely offline on headless CI nodes. They don't care about mavenLocal() locks or staging delays on remote repositories.
2. Absolute Lifecycle Parity: Whether your enterprise application is running on Gradle 7.6.6, Gradle 8.7, or cutting-edge Gradle 9.x matrices, the configuration-time signatures match perfectly without crashing.
3. Pristine Daemon State: Tool binaries are contained strictly within the worker's parent context, keeping the global Gradle daemon's main classpath clean and unpolluted.

In Part 3 of our Gradle Under the Hood series, we will look at Thin Archives and discuss why narrowing dynamic dependency propagation to exclusive compileOnly scopes is crucial to prevent runtime classpath pollution in enterprise Jakarta EE application servers.

---

What are your experiences dealing with classloader leakage or configuration API breaks between different Gradle lifecycles? Let's discuss in the comments below!

🔗 Check out the JPersist open-source architecture: https://github.com/jpersist/persistence