NOTE: To achieve build-cache node replication, you need to have an instance of Gradle Enterprise running.
In this post, I will share our experience at Premise Data using build-cache node replication to serve our distributed teams across the globe. I will also share the inner workings of the Gradle build script solution for replicated node selection.
But before we delve into build-cache node replication, let us do a brief recap of the gradle build cache.
Gradle Build Cache
The Gradle build cache is a caching mechanism that aims to save the developer's time by reusing outputs produced by other builds.
The build cache works by storing build outputs(locally or remotely) and allowing builds to fetch these outputs from the cache when it is determined that inputs have not changed, avoiding the expensive work of regenerating them.
The Build Cache is a build acceleration technology that enhances build performance. The benefits of the build cache are as follows:
- Faster build times lead to faster feedback
- Faster feedback leads to better developer productivity
- Better developer productivity ships features quicker
Build Cache Nodes
The Gradle build-cache node software is a freely available HTTP remote cache server for Gradle’s build caching functionality.
The build-cache node can be used without a Gradle Enterprise installation with restricted functionality.
Build-cache nodes can optionally be connected with Gradle Enterprise for centralized management and monitoring, and to enable replicating cache entries between multiple nodes.
By connecting remote nodes to a Gradle Enterprise instance, you are able to configure them centrally from Gradle Enterprise and have them replicate entries from the default built-in cache node.
The installation and operation of remote build cache nodes is documented in the Build Cache Node User Manual and the process of connection of the cache nodes to Gradle Enterprise is documented in Connecting with Gradle Enterprise.
Replication
Replication allows users to use a cache node that they have a better network link to, while reusing artifacts from the main cache node that is further away. This is particularly effective for geographically distributed teams.
In our case, the main cache node is located in US, Seattle and we have replicated nodes to serve our developers in other places around the globe.
The graphic below shows our team distribution by highlighting the developer locations:
The replication settings for each node can be configured via the Nodes configuration page in Gradle Enterprise. They can not be configured for remote nodes via the remote node’s user interface or configuration file.
Tip #1
A typical arrangement is to have continuous integration builds push to a default built-in cache node on a local network, and have other nodes used by developers in different locations,( ideally on their local network), use it as their replication source.
The replication technology can be used to separate cache artifacts, distribute the load, and improve build performance by having a better network link between the developer's build and the node.
NOTE: The bandwidth and latency of the network link between a build and a build cache significantly affect build performance.
Tip #2
We keep an eye on the network bandwidth and latency using the Gradle Doctor plugin that outputs the remote build-cache benchmark report whenever our builds run.
Value Proposition
The main value proposition is that by replicating cache nodes, your developer's local build times will significantly be reduced. The effect will be faster feedback cycles & a highly satisfying developer experience.
Connecting to Gradle Enterprise
The process of connecting a remote cache node to your Gradle Enterprise installation is pretty straightforward.
(1) Create a record for the node in Gradle Enterprise
Visit /cache-admin on your Gradle Enterprise dashboard, and select Nodes from the left menu. In the Remote nodes > Create new node section, enter the name for your node and click Create new node.
In this example, we will assume we have a new engineer joining the team from Honduras.
The Honduras node will now be listed in the Existing nodes section.
Each node is assigned a key and a secret. The node needs to be configured with the key and secret.
The secret is only viewable for 5 minutes after node creation. If the node secret is lost, use the regenerate function to issue a new secret which will then be viewable for 5 minutes.
See screenshot below:
(2) Gradle Enterprise Registration
We will now proceed to register and connect the build-cache node with our Gradle Enterprise installation. This registration enables centralized monitoring and cache entry replication which are extra features.
The registration can be configured via the web interface or via the config file. In our case, we will configure it via the web interface.
(i) Web Interface Registration
Open the web interface of your build cache node on your browser. Focus on the Gradle Enterprise section on the web interface and expand the Settings dropdown.
Configure your node with the details of the Gradle Enterprise server to connect with. They include the following:
Gradle Enterprise server ~ The URL of the Gradle Enterprise instance we desire to connect to.
My address ~ The public address of this build-cache node.
Key ~ The identifying key of this node. This is obtained from Gradle Enterprise via the replication step above.
Secret ~ The secret value for this node. This is obtained from Gradle Enterprise via the replication step above.
Click the Save button. If your setup is done correctly, your node should successfully connect to the Gradle Enterprise instance and should be available for replication.
Gradle Build Script Solution For Replicated Node Selection
Now that we have our cache nodes setup, its time to explain our build script solution. The script enables a developer to use a cache node that they have a better network link to.
The script is available in this Android project repo. The project has been setup to generate build scans and leverage the build-cache technology.
The 3 main files to focus on are:
- build-cache-node-config.groovy
- build.gradle(project level)
- settings.gradle
(i) build-cache-node-config.groovy
This file contains the title and URL of all our build-cache nodes that are connected to our Gradle Enterprise instance.
environments {
default_node {
nodeTitle = '# Default Build Cache Node'
nodeUrl = 'https://gradle-enterprise.your-domain.com/cache/'
}
east_africa_node {
nodeTitle = '# East Africa Build Cache Node'
nodeUrl = 'https://east.xx.yyy.zzz/cache/'
}
west_africa_node {
nodeTitle = '# West Africa Build Cache Node'
nodeUrl = 'https://west.xx.yyy.zzz/cache/'
}
us_west_node {
nodeTitle = '# US West Build Cache Node'
nodeUrl = 'https://us-west.xx.yyy.zzz/cache/'
}
honduras_node {
nodeTitle = '# Honduras Build Cache Node'
nodeUrl = 'https://honduras.xx.yyy.zzz/cache/'
}
}
Based on the node a user selects, the title and URL path will be output to a build-cache-node-config.properties file that is created during the node selection process.
(ii) build.gradle
This is the project level gradle file. It contains a code region responsible for generating the build-cache-node-config.properties that contains the build-cache node selected by the developer.
// region Build Cache Node Replication Setup
def loadBuildCacheNodes() {
def node_title = hasProperty('node') ? node: 'default_node'
println "Current Build Cache Node: " + node_title
def configFile = file('build-cache-node-config.groovy')
def config = new ConfigSlurper(node_title).parse(configFile.toURL())
project.ext.config = config
}
task setBuildCacheNode {
doLast {
if (project.hasProperty('node')) {
loadBuildCacheNodes()
new File(rootDir, "build-cache-node-config.properties").text =
""" $config.nodeTitle \n replicated_build_cache_node=$config.nodeUrl"""
println "nodeTitle: $config.nodeTitle"
println "nodeUrl: $config.nodeUrl"
} else {
// empty
}
}
}
task setEastAfricaBuildCacheNode() {
group = "build setup"
description = "Configures the remote build cache url for devs in East Africa"
doFirst {
project.ext.set("node", "east_africa_node")
}
finalizedBy "setBuildCacheNode"
}
task setWestAfricaBuildCacheNode() {
group = "build setup"
description = "Configures the remote build cache url for devs in West Africa"
doFirst {
project.ext.set("node", "west_africa_node")
}
finalizedBy "setBuildCacheNode"
}
task setUSABuildCacheNode() {
group = "build setup"
description = "Configures the remote build cache url for devs in the United States"
doFirst {
project.ext.set("node", "us_west_node")
}
finalizedBy "setBuildCacheNode"
}
task setHondurasBuildCacheNode() {
group = "build setup"
description = "Configures the remote build cache url for devs in Honduras"
doFirst {
project.ext.set("node", "honduras_node")
}
finalizedBy "setBuildCacheNode"
}
//endregion
loadBuildCacheNodes() - This function loads the nodes available in the build-cache-node-config.groovy file and passes it over to the setBuildCacheNode task.
setBuildCacheNode - This task accepts the user's selected node as input and creates a new build-cache-node-config.properties file containing the build cache node title and URL.
setEastAfricaBuildCacheNode, setWestAfricaBuildCacheNode, setUSABuildCacheNode, setHondurasBuildCacheNode - These tasks set the required node as per the highlighted region and passes this node selection as input to the setBuildCacheNode task.
(iii) settings.gradle
In the remote(HttpBuildCache) closure contained in this gradle file, we add some logic to determine the replicated cache node URL that our project will pull artifacts from.
remote(HttpBuildCache) {
// Create a variable called buildCacheNodesPropertiesFile and initialize it to your
// build-cache-nodes.properties file, in the rootProject folder.
def buildCacheNodesPropertiesFile = new File("build-cache-node-config.properties")
if (!buildCacheNodesPropertiesFile.exists()) {
url = "https://gradle-enterprise.your-domain.com/cache/"
} else {
// Initialize a new Properties() object called buildCacheNodesProperties
def buildCacheNodesProperties = new Properties()
// Load the build-cache-nodes.properties file into the buildCacheNodesProperties object.
buildCacheNodesProperties.load(new FileInputStream(buildCacheNodesPropertiesFile))
if (buildCacheNodesProperties != null
&& buildCacheNodesProperties.containsKey('replicated_build_cache_node')) {
// Set the dev specified build cache node url
url = buildCacheNodesProperties['replicated_build_cache_node']
} else{
// Set the default build cache node url
url = buildCacheNodesProperties['default_build_cache_node']
}
}
...
}
The above logic basically reads the contents of the build-cache-node-config.properties file and determines whether our project leverages the selected replicated node or whether it falls back to the default built-in cache node.
Build Cache Node Selection Process
Now that we have an understanding of Gradle Enterprise, Build Cache Node Replication, and the inner workings of our Gradle build script, we will now go through the build cache node selection process.
We will achieve this from our IDE. Our IDE of choice is Android Studio.
There are 4 build-cache nodes for the 4 main regions our developers are currently located in:
east_africa_nodefor East Africa.west_africa_nodefor West Africa.us_west_nodefor US West Coast.honduras_nodefor Honduras. (The node we created in this blog post)
In Android Studio, select the Gradle tab and open the build setup group of tasks. This should display the build cache node tasks available per region.
Click on your preferred gradle task (setHondurasBuildCacheNode) and this action will result in the following output under the Build tab in your IDE.
4:19:56 PM: Executing task 'setHondurasBuildCacheNode'...
Executing tasks: [setHondurasBuildCacheNode] in project /Projects/Android/gradle-build-cache-node-replication
> Task :setHondurasBuildCacheNode
> Task :setBuildCacheNode
Current Build Cache Node: honduras_node
nodeTitle: # Honduras Build Cache Node
nodeUrl: https://honduras.xx.yyy.zzz/cache/
BUILD SUCCESSFUL in 552ms
2 actionable tasks: 2 executed
Publishing build scan...
https://gradle.com/s/s3x2bmklo2vjc
4:19:58 PM: Task execution finished 'setHondurasBuildCacheNode'.
This task action will generate a build-cache-node-config.properties file at the root level of your project.
NOTE: Changes to this file are not tracked by source control.
Also, if you open the generated build scan in your browser, you will confirm that our project is leveraging the Honduras cache node.
NOTE: The 4 Gradle tasks can also be run directly from the command line:
./gradlew setUSABuildCacheNode
./gradlew setEastAfricaBuildCacheNode
./gradlew setWestAfricaBuildCacheNode
./gradlew setWestAfricaBuildCacheNode
That's All Folks
It is my hope that you found this useful. If there's anything specific I haven't mentioned above that you think other readers will benefit from, feel free to mention it in the comments section below.
Enjoy!
Top comments (2)
Great writeup Roger!
Thanks Nate :)