Gradle builds don’t maintain themselves. They require proper care and feeding (nutritious food rather than the chips and soda most of us give them), else they will get way out of hand, demanding ever more memory and CPU cycles and precious, precious attention. Your build requires active management, just like any other part of your codebase. And just like with your application source, there exist automated tools to help make it a bit easier.
In this article, we will take a look at the Dependency Analysis Gradle Plugin, of which I am the author. This plugin is uniquely designed to solve several build problems in the Android & JVM build ecosystems such as telling you which (if any) of your dependencies are unused and can be removed.
To put it simply: if you follow the advice of the plugin, you will end up with an optimized dependency graph, regardless of project complexity.1
Features
The plugin answers these questions:
- Do I have any unused dependencies in my build?
- Do I have unused annotation processors in my build?
- Am I declaring my dependencies correctly? (
api,implementation,compileOnly,testImplementation...) - Am I directly using transitive dependencies that might be best declared directly?
- Do I have plugins applied that are superfluous?
The plugin can provide thorough answers to these questions, thanks to its deep integration with both Gradle and the Android Gradle Plugin (AGP). In particular, it is variant-aware, so it also knows if a dependency should be on implementation or (e.g.) debugImplementation.
From the user’s perspective, the plugin answers these questions in two compatible ways: first, with detailed, human-readable console output; and second, with machine-readable json, which is intended for automation that is built on top of this plugin.
This plugin works with Java, Kotlin, and Android projects.
How to use it
The quickest way to get started is to simply apply the plugin
// root build.gradle[.kts]
plugins {
id("com.autonomousapps.dependency-analysis") version "0.56.0"
}
and then execute
$ ./gradlew buildHealth
from the command line. The Github repository has extensive documentation, including a wiki, that explains the many configuration options (some of which we’ll touch on later). However, all of those options are indeed optional, and the plugin is meant to be useful out-of-the-box and with zero additional configuration.
Special thanks
Before we dig in, I want to offer special thanks to Stéphane Nicolas at Square and Zac Sweers at Slack. The former has provided detailed performance data which I’ll reference below, while the latter has helped me improve the automation story significantly. Both have contributed code and precious time in long discussions about design of and goals for the plugin. Thank you!
Benefits
We’ll examine each of the use-cases in turn, discussing the benefits of caring as we go.
Removing unused dependencies
I’ve had an interest in this subject ever since I learned about the Gradle Lint Plugin way back in June 2016 (you know, before the timeline split and we all ended up in the worst version). Unfortunately, that plugin never became Android-compatible, so I ultimately decided to solve this problem for myself.
The benefits of removing unused dependencies are two-fold:
- Keep your build script manageable and maintainable. Really understanding what your project needs to be built, rather than just keeping everything + the kitchen sink.
- Build performance, naturally.
I hope the first is self-explanatory, but for the second at least we have some hard data on the benefits of removing unused dependencies. Let’s set the stage. On Square’s famous monolith, with nearly 3 000 Gradle modules (aka projects aka subprojects), buildHealth was invoked on a subsection of the full module graph, and there found about 3 000 unused dependencies (2 000 between modules and 1 000 external). In a series of experiments run with the help of gradle-profiler, Square found that they could improve both IDE sync time (a critical indicator) and build time by removing some unused dependencies. For IDE sync, in one experiment they removed just 20% of the dependencies that were identified as unused; this resulted in an improvement of 10s, or 3.7%. For total build time, they attempted to identify “bottleneck” modules via an ad hoc methodology, and in those bottleneck modules removed some dependencies. In this case, they saw build performance gains of 4–17%, depending on scenario / build target tested.
Square hypothesizes that these performance gains are the result of better parallelization, reduced memory consumption, less garbage collection, and improved resolution speed for external dependencies. While their builds benefit from a better parallelization when removing unused inter-modules dependencies, they also observed that eliminating unused external dependencies had more impact on Android Studio sync.
While these data are limited, they are not anecdotal. The gradle-profiler tool is very effective at developing reproducible results in the build domain. Given the clear benefits of removing unused dependencies, having a tool that automates their discovery must be seen as invaluable.
Removing unused annotation processors
A special type of unused dependency analysis is detecting unused annotation processors. The plugin is able to determine if an annotation processor is unused, and if so tell you about it. Even a redundant annotation processor incurs a cost; in particular, the processor’s init() function will still be called, and in some cases this can be expensive. In addition to that, kapt itself (if you’re using Kotlin) will generate stubs even if they serve no ultimate purpose. If you remove all unused processors and find you have none left, you can also remove the Kapt Gradle plugin, improving build performance even more. One should not apply kapt — or any plugin, really — unless absolutely necessary.
Declaring dependencies correctly
Thanks to a detailed bytecode analysis of producer and consumer code, the plugin is able to compute the ABI (binary API) of both Java and Kotlin modules, and also determine if a dependency is only required for compilation (and not at runtime). Thanks to this detailed analysis, it knows whether a dependency should be:
implementationapicompileOnly- Android-specific variants thereof
- and of course
annotationProcessororkapt
Declaring these correctly is not merely academic. For library authors in particular, it is crucially important to get right: failing to declare an api dependency can lead to compilation failures for users of your library. Over-declaring api dependencies will increase the size of the compilation classpath, which has performance implications, while also increasing your API surface in subtle, poorly-understood ways. Declaring a dependency only required for compilation as implementation instead of compileOnly means it will get bundled into your application, increasing binary size without benefit.
Undeclared transitive dependencies
One of the plugin’s features is that it tells you if you are using a dependency you have not declared; that is, a dependency that is brought in transitively via another dependency that you have declared. The fact that this can happen is down to one of two things:
First, in Maven dependency management (which most Java libraries still rely on), there is no distinction made between api and implementation. If a library you’re using only has a pom.xml (cf Gradle Module Metadata), then probably most or all of its dependencies will be on the compile scope, which is roughly equivalent to Gradle’s api configuration.
Second, for libraries published with Gradle Module Metadata (including inter-module dependencies in a large multi-module build), it is not unusual to find the api configuration to be over-used. This might happen if the author simply didn't know which dependencies were part of the ABI and wanted to play it safe, or, in a multi-module project, certain "common" dependencies might be declared api as a convenience for downstream consumers. The problem with this anti-pattern is that the build maintainer or library author could change a dependency from api to implementation without changing the actual ABI (and therefore “safely”), but nevertheless break downstream builds who required those dependencies for compilation (in many cases without even realizing it). So we find that a simple upstream dependency change (supposedly transparent!) has broken the build.
I argue that this feature of the plugin forces you to think about your dependencies more clearly, which is a good thing. Your project's API is more than just source — it's also your dependencies.
That said, I must admit that this feature has always been the most controversial. If you fall into the camp of not caring, be of glad heart, for the plugin is highly configurable. (More on that below.)
Redundant plugins
The final feature provided by the Dependency Analysis Plugin is that it can tell if certain other common plugins are redundant (as in, completely unnecessary and providing no value to your project). All plugins imply a build cost (some more than others), so it’s worth it to remove those you don't need. As of the time of writing, this includes:
-
kapt— if there are no used annotation processors, this plugin is redundant. -
java-library— ifkotlin(“jvm”)is also applied, and the project contains only Kotlin code, this plugin is redundant. -
kotlin(“jvm”)— ifjava-libraryis also applied, and the project contains only Java code, this plugin is redundant.
In the future, I would like to be able to detect if com.android.library can be replaced with either java-library or kotlin(“jvm”), which would be true if no Android-specific features were in use. This would improve build performance, as JVM projects are much simpler to build than Android projects.
Configuration
The plugin is highly configurable. The wiki is the best place to go to see all the options, but some of the most important can be sketched out here in brief.
We discussed earlier the contentious subject of declaring transitive dependencies, and mentioned that if you don’t care about such things, we got your back. Consider the following:
// root build.gradle[.kts]
dependencyAnalysis {
issues {
all {
onUsedTransitiveDependencies {
severity(“ignore”)
}
}
}
}
This bit of DSL means that, for all modules, on any used transitive dependency, ignore it. You may also configure each project individually. Please see this section of the wiki for more information.
Automation, or post-processing
The plugin has support for post-processing the results of the analysis as the foundation for automated correction of any issues discovered. While it is not yet in full production, Slack is currently doing some automation over the results of the dependency analysis. Two features of the plugin support this automation: all results are stored on the filesystem, most in json format, with model classes public and on the classpath; and an abstract task class that you can extend and register with the plugin, which receives as an input the plugin's terminal output, and runs after the final built-in task. You can read more about it on the wiki. A longer discussion is outside the scope of this article.
How does it work?
The short answer is many thousand lines of Kotlin code and a number of complex algorithms, and the long answer will be its own full-length post!
Performance and reliability
One goal I had when writing this plugin was to follow best-practices for plugin development. As such, every task contributed by the plugin correctly declares its inputs and outputs, and almost all are cacheable via Gradle's build cache (and for those that aren't, this is intentional). Furthermore, many tasks make use of the Worker API, which means they can run in parallel with other Workerized tasks. As a consequence, the plugin is very performant, even in the face of its extensive analysis.
The plugin is also thoroughly tested. There are well over 300 end-to-end tests that run against a matrix of supported AGP and Gradle versions (many tests also run against pure JVM projects, written in Java or Kotlin). Since the plugin supports AGP from 3.5.3 (including alphas and betas!) and Gradle from 6.0.1, you can have confidence that the plugin will just work for your project.
Conclusion
To recapitulate, using the plugin is as simple as applying it
// root build.gradle[.kts]
plugins {
id("com.autonomousapps.dependency-analysis") version "0.56.0"
}
and then executing
$ ./gradlew buildHealth
from the command line. It is zero configuration out-of-the-box. It is fairly well-optimized and can complete an analysis of even Square's gigantic monolith in under 20 minutes on a CI worker (this basically uses All the Memory.) Since very few projects approach the level of complexity that Square has, it's likely that you won't find performance to be an issue.
For three lines of code in a build script and one command-line execution that you could run only on CI jobs, you'll generate tremendous value — in understanding, in correctness, and in performance.
Further reading
Please also check out the Gradle Doctor plugin, which analyzes your build as it runs to give you immediate performance feedback.
Endnotes
1 This is a bit of an exaggeration. The plugin will enable you to optimize your dependency graph as your project is currently constituted. It does not (yet?) provide advice on, for example, how to split large modules into smaller ones.
Top comments (6)
I'm very keen on trying out this plugin. However, one can't help but wonder if it will be of any use for Gradle-built Kotlin-based Spring Boot project. Have you had such use-case in mind writing this solution?
I know of at least one Spring Boot user who uses it. The issue is that the Spring Boot plugin (for some reason?) disables the jar task when you apply it. My plugin requires the jar task to execute so it can extract information from it. The workaround is to manually enable your jar task. I can't provide a more hands-off solution without first understanding why Spring Boot does this. (And since I'm not a Spring Boot user, I need people like you to tell me :) )
The latest version of the plugin, 0.53.0, provides support for Spring Boot. The plugin no longer relies on the jar task (just the .class files), so there's no longer any issue.
This is a great post, however, I would like to see more examples (e.g. outputs of the plugin and the samples of the build configuration that cause the issues).
That's fair, but the post was already fairly long :)