Gradle Under the Hood: Fighting Incremental Cache Misses in Bytecode Processing

If you have ever written a custom Gradle task to analyze or process compiled bytecode (like custom linters, schema validators, or metadata generators), you have probably encountered a hidden, silent performance killer: the aggressive compiler erasure.

In this first article of our Gradle Under the Hood series, we’ll explore how we re-engineered the incremental tracking of JPersist (a zero-boilerplate JPA build automation suite) to achieve bulletproof UP-TO-DATE execution states, even when the Java compiler tries to destroy your task's cache.

The Naive Approach to Bytecode Tasks

When building a task that processes compiled .class files, the most intuitive setup is to declare the compiler's output directory as your task's @InputDirectory:

abstract class MyBytecodeTask extends DefaultTask {
    @InputDirectory
    abstract DirectoryProperty getCompiledClassesDir()

    @TaskAction
    void execute(InputChanges inputChanges) {
        // Process altered files...
    }
}

You then wire it in your plugin by binding it to the JavaCompile task:

task.getCompiledClassesDir().set(compileJavaTask.flatMap { it.destinationDirectory })

Why this fails in the real world:
Gradle has a magnificent incremental build engine. However, JavaCompile is highly defensive. When a developer alters even a single line of Java source code, JavaCompile often wipes out its entire destination directory (build/classes/java/main) before rebuilding to guarantee no stale classes remain.

For your downstream bytecode task, this folder deletion is catastrophic. Even though you are using Gradle's InputChanges API, the engine sees that the root directory was destroyed and recreated. It panics, concludes that the historical file snapshot graph is non-linear, and forces inputChanges.incremental to evaluate to false. Your task is forced into a heavy, full-pass execution loop on every minor code tweak.

The Solution: Source-Driven Incremental Tracking

To make our query linters and schema validators truly incremental and immune to compiler erasures, we shifted our architectural paradigm: We stopped tracking the volatile compiler output folder and started tracking the stable source code repository layout instead.

Instead of listening to the .class directory, our tasks now listen to sourceSet.java.sourceDirectories as a flat @InputFiles collection wrapper:

abstract class ValidatePersistenceSchema extends DefaultTask {
    @Incremental
    @InputFiles
    @PathSensitive(PathSensitivity.RELATIVE)
    abstract ConfigurableFileCollection getSourceFiles()

    @InputDirectory // Kept as standard non-incremental input
    abstract DirectoryProperty getCompiledClassesDir()
}

Inside our plugin wiring loop, we feed it lazily while filtering out any intermediate or generated build directories:

task.getSourceFiles().from(project.provider {
    sourceSet.java.sourceDirectories.asFileTree.matching {
        exclude { element ->
            element.file.absolutePath.replace('\\', '/').contains(project.layout.buildDirectory.get().asFile.absolutePath.replace('\\', '/'))
        }
    }
})

Mapping Source Deltas to Bytecode Outputs

Because source directories are managed by the developer and are never aggressively wiped out by a compiler task, Gradle's file system snapshot engine maintains a perfect, unbroken lineage of mutations.

When a developer modifies an entity, InputChanges accurately catches exactly one modified .java file. Inside our @TaskAction, we intercept this .java path, compute its relative package structure, and translate it on-the-fly to find the corresponding compiled .class file inside the freshly compiled folder:

@TaskAction
void validate(InputChanges inputChanges) {
    List<String> affectedEntities = []

    if (inputChanges.incremental) {
        inputChanges.getFileChanges(getSourceFiles()).each { change ->
            if (change.file.name.endsWith('.java')) {
                String relativePath = change.normalizedPath
                // Translate com/example/model/User.java -> com.example.model.User
                String className = relativePath.substring(0, relativePath.lastIndexOf('.java')).replace('/', '.')

                if (change.changeType != ChangeType.REMOVED) {
                    affectedEntities.add(className)
                }
            }
        }

        // Fast-Exit Pass: If no active entities were touched, skip heavy DB bootstrapping!
        if (affectedEntities.isEmpty()) return
    } else {
        affectedEntities.addAll(fullDiscoveredEntityList)
    }

    // Submit isolated work wrapper with precisely filtered entity subsets...
}

The Engineering Win

By decoupling task delta tracking from the volatile compilation output graph, we achieved major engineering victories for the JPersist v1.8.0 release pipeline:

1. Immunity to Directory Purges: Even if JavaCompile erases and rewrites 5,000 class files, our tasks compare the stable source metadata signatures. If only 1 file changed, we only process 1 class binarized stream via ASM.
2. The Fast-Exit Optimization: If a developer edits a standard Service, Controller, or Utility class that doesn't hold JPA annotations, our tasks process the change stream in under 5 milliseconds and exit before ever spawning database connections or heavy classloaders.

Stay tuned for Part 2, where we’ll dive into ClassLoader Harvesting and how we forced isolated Gradle workers to run seamlessly on headless CI nodes without ever hitting network lookups or requiring mavenLocal() fallbacks!

---

What are your thoughts on handling aggressive compiler tasks in custom Gradle plugins? Let's discuss in the comments below!

🔗 Check out the JPersist open-source architecture: https://github.com/jpersist/persistence