Implicit Dependencies Management with Nx: A Practical Guide through Real-World Case Studies

In this article, I will present two concrete cases of implicit dependencies and show you how to configure your Nx workspace and projects to deal with them.
You can find the repository that illustrates those cases on GitHub.

Before we dive into the cases, let’s take a moment to understand how Nx represents dependencies and how it can help us work with implicit dependencies.

In the context of a workspace, Nx maintains a graph of all projects it contains, including internal and external dependencies. It does its best to deduct relationships by continuously analyzing the workspace components, such as the source code, the projects' configuration, the installed dependencies, etc. All being explicit dependencies. For the rest - the implicit dependencies that only a human could be aware of - Nx offers a way to declare them so they can also be part of the graph.

We can use the following graph extract to understand its composition:

{
  "graph": {
    "nodes": {
      "ts-interfaces": {
        "name": "ts-interfaces",
        "type": "lib",
        "data": {
          "root": "libs/shared/ts-interfaces",
          "name": "ts-interfaces",
          "targets": {
            // ...
          },
          "$schema": "../../../node_modules/nx/schemas/project-schema.json",
          "sourceRoot": "libs/shared/ts-interfaces/src",
          "projectType": "library",
          "implicitDependencies": ["schemas"],
          "tags": ["platform:js", "type:core"]
        }
      },
      "schemas": {
        "name": "schemas",
        "type": "lib",
        "data": {
          "root": "libs/shared/schemas",
          "name": "schemas",
          "targets": {
            // ...
          },
          "$schema": "../../../node_modules/nx/schemas/project-schema.json",
          "sourceRoot": "libs/shared/schemas/src",
          "projectType": "library",
          "tags": ["platform:shared", "type:core"],
          "implicitDependencies": []
        }
      },
      "coffee-dealer": {
        "name": "coffee-dealer",
        "type": "app",
        "data": {
          "root": "apps/coffee-dealer",
          "name": "coffee-dealer",
          "targets": {
            // ...
          },
          "$schema": "../../node_modules/nx/schemas/project-schema.json",
          "sourceRoot": "apps/coffee-dealer/src",
          "projectType": "application",
          "tags": ["type:app", "platform:node"],
          "implicitDependencies": []
        }
      }
    },
    "dependencies": {
      "ts-interfaces": [
        {
          "source": "ts-interfaces",
          "target": "schemas",
          "type": "implicit"
        }
      ],
      "schemas": [],
      "coffee-dealer": [
        {
          "source": "coffee-dealer",
          "target": "ts-interfaces",
          "type": "static"
        }
      ]
    }
  }
}

Note

• This graph is generated by running nx graph --file=output.json && cat output.json in the repository root folder.
• If you are unfamiliar with Nx, you might want to head to Nx’s documentation to understand the benefits of the project graph and the mental model.

Under graph.nodes, each node represents a project in the workspace. The graph.dependencies include the edges defining the relationship between nodes/projects. The type property can be static or implicit, depending on whether the dependency is explicit or implicit.

Case 1: The Libraries Hidden Bridge

This case occurred in an integrated repository composed of multiple apps and libraries. I reproduced a similar situation with the following setup:

• coffee-dealer, is a NestJS application importing TS interfaces from ts-interfaces
• ts-interfaces, is composed of auto-generated Typescript interfaces from schemas
• schemas, is composed of JSON schemas exported in a TS file

You can see the current relation between the projects in the project graph:

And the tasks graph for the three projects:

What is wrong with these graphs?

The schemas library is not linked to the ts-interfaces library, meaning that when building the application with nx run coffee-dealer:build:

1. The schemas:bundle task will not be included in the task pipeline, inducing the risk of using outdated schemas while generating the Typescript interfaces.
2. The schemas:bundle and ts-interfaces:bundle tasks runner is not caching inputs, so Nx cannot check whether the source definitions have changed at the next bundle run.

In the following sections, I will further present the two libraries and show you how to solve this issue.

schemas

Link to the library

These JSON schemas represent the domain entities and value objects; in other words, they are a source of truth and can be derived into types for different programming languages (Go struct, Typescript interfaces … ).
The JSON schema definitions organization follows this good advice so that each definition represents a single domain entity or value object, making work easier during development. Before being shared, the definitions are bundled and exported into larger schemas representing business operations.

The logic to bundle the JSON schemas is located outside the src folder, in the internals folder, to avoid including those scripts and schemas in the build output. To still benefit from Typescript in our IDE, a separate config is created under tsconfig.editor.json and referenced in tsconfig.json to include the internals folder.

// libs/schemas/tsconfig.editor.json
{
  "extends": "./tsconfig.json",
  "compilerOptions": {
    "outDir": "../../dist/out-tsc",
    "declaration": true,
    "resolveJsonModule": true,
    "esModuleInterop": true,
    "types": ["node"]
  },
  "include": ["src/**/*.ts", "internals/**/*.ts"],
  "exclude": ["jest.config.ts", "test/**/*.spec.ts", "test/**/*.ts"]
}

Now let’s zoom in on the project.json

• It contains an extra namedInputs, which references the content of internals, allowing Nx to use this as a cache input for a task
• In the build task, I added a dependency on the bundle task, meaning that before compiling the Typescript code, Nx will execute the bundle task
• The bundle task executes the code containing the logic to bundle schemas into the src/lib folder. The cache property is set to true to enable task runner caching. The inputs use the internals named input, and the outputs reference the glob {projectRoot}/src/lib/*.json, which matches all bundled schemas.

Note
If the cache is hit because the input has not changed, Nx will restore the existing outputs.

// libs/schemas/project.json
{
  "name": "schemas",
  "$schema": "../../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "libs/shared/schemas/src",
  "projectType": "library",
  "namedInputs": {
    "internals": ["{projectRoot}/internals/*.{json,ts}"]
  },
  "targets": {
    "build": {
      "executor": "@nx/js:tsc",
      "outputs": ["{options.outputPath}"],
      "options": {
        "outputPath": "dist/libs/shared/schemas",
        "main": "libs/shared/schemas/src/index.ts",
        "tsConfig": "libs/shared/schemas/tsconfig.lib.json",
        "assets": [
          "libs/shared/schemas/*.md",
          "libs/shared/schemas/src/lib/*.{json,js}"
        ]
      },
      "dependsOn": ["bundle"]
    },
    "bundle": {
      "executor": "nx:run-commands",
      "cache": true,
      "inputs": ["internals"],
      "outputs": ["{projectRoot}/src/lib/*.json"],
      "options": {
        "command": "npx ts-node libs/shared/schemas/internals/bundle.ts",
        "cwd": "."
      }
    }
  },
  "tags": ["platform:shared", "type:core"]
}

ts-interfaces

Link to the library

The ts-interfaces library is composed of auto-generated Typescript interfaces. A custom script, located in internals for the same reason as the schemas library, loads the JSON schemas using the File system API instead of imports to avoid including schemas dependencies to generate the interfaces.

The drawback is that Nx can't detect the dependency between schemas and ts-interfaces; let’s see how to declare the relationship between those two libraries by examining the project.json. It contains a configuration similar to the schemas project with the following noticeable additions:

• The implicitDependencies declare a dependency on the schemas library, which creates the missing link in the project graph
• In the bundle target, dependsOn is now set to ^bundle, meaning all bundle tasks from the dependencies will run first when calling nx run ts-interfaces:bundle
• The bundle.inputs now reference the internals named input from the current library
• The bundle.outputs now reference the path (glob) of the generated Typescript interfaces

// libs/ts-interfaces/project.json
{
  "name": "ts-interfaces",
  "$schema": "../../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "libs/shared/ts-interfaces/src",
  "projectType": "library",
  "namedInputs": {
    "internals": ["{projectRoot}/internals/*.ts"]
  },
  "implicitDependencies": ["schemas"],
  "targets": {
    "build": {
      "executor": "@nx/js:tsc",
      "outputs": ["{options.outputPath}"],
      "options": {
        "outputPath": "dist/libs/shared/ts-interfaces",
        "main": "libs/shared/ts-interfaces/src/index.ts",
        "tsConfig": "libs/shared/ts-interfaces/tsconfig.lib.json",
        "assets": ["libs/shared/ts-interfaces/*.md"]
      },
      "dependsOn": ["bundle"]
    },
    "bundle": {
      "executor": "nx:run-commands",
      "cache": true,
      "dependsOn": ["^bundle"],
      "inputs": ["^internals"],
      "outputs": ["{projectRoot}/src/lib/*.ts"],
      "options": {
        "command": "npx ts-node --project libs/shared/ts-interfaces/tsconfig.editor.json libs/shared/ts-interfaces/internals/bundle.ts",
        "cwd": "."
      }
    }
    // ...
  },
  "tags": ["platform:js", "type:core"]
}

After applying these changes, the project graph contains the expected relationship between the schemas and ts-interfaces libraries.

And the tasks graph shows that the schemas:bundle task is now part of the coffee-dealer:build pipeline.

Enable cache for the bundle tasks

There are two ways to enable caching of tasks :

1. on a project basis by setting the cache property to true and the inputs to cache in the target, as shown previously for the schemas and ts-interfaces libraries
2. globally, by setting the bundle default values in targetDefaults and declaring internals in namedInputs in nx.json

// nx.json
{
  "$schema": "./node_modules/nx/schemas/nx-schema.json",
  "targetDefaults": {
    // ...
    "bundle": {
      "cache": true,
      "dependsOn": ["^bundle"],
      "inputs": ["^internals"]
    }
    // ...
  },
  "namedInputs": {
    // ...
    "internals": [
      "{projectRoot}/internals/**/*.[jt]s",
      "{projectRoot}/internals/**/*.json"
    ]
    // ...
  }
}

Tips

The technique above allows you to declare a global rule for the cache and the named inputs. It is useful when you have many projects sharing the same configuration. This can also be applied to executors, such as @nx/jest:jest or @nx/eslint:lint.
Have a look at the Nx docs, to learn how to further tweak the cache rules

Our case is now closed, and we can move on to the next one.

Case 2: The software developer's optimism

This story is another case of implicit dependencies impacting developers' experience. I reproduced a similar situation with the following setup:

• coffee-dealer, is a NestJS application serving an HTTP API on port 3000
• coffee-dealer-cli, is a CLI tool to interact with coffee-dealer through its HTTP API
• coffee-dealer-e2e, is an end-to-end test suite sending HTTP requests to coffee-dealer

In this case, the developers would like to ensure that when they work locally on the coffee-dealer application with the watch mode enabled, the coffee-dealer-cli and coffee-dealer-e2e are always running against the latest build of the coffee-dealer application.

You can see the current relation between the projects in the project graph:

What can we deduct from this graph?

1. We can see an implicit dependency between coffee-dealer and coffee-dealer-e2e, created by default by the NestJS generator plugin, and no dependency between coffee-dealer and coffee-dealer-cli. There is no task dependency between the coffee-dealer-e2e:e2e task and the coffee-dealer:serve task, meaning that the end-to-end tests could run before the application is ready. It is the same for the coffee-dealer-cli:run task, which could run before the application's API is available. With Nx, it is currently impossible to declare a task dependency for a long-running task, such as serve, so we must find an alternative solution. See related issues: nx issue #5570 and nx issue #3748.
2. When running the application with nx run coffee-dealer:serve, it starts with the watch mode enabled. The watcher does not instantly detect changes; there is a delay (which depends on the compiler or task manager implementation and/or configuration) between the file(s) change detection and the build completion. It means there is a small window where the CLI or the end-to-end tests could run against a previous build version.

coffee-dealer-e2e

Link to the application

To dramatically decrease the risk of encountering the abovementioned issues, we can :

• Create a new task check-port and add it to dependsOn of the tasks that should check whether the coffee-dealer port is open before running the end-to-end tests
• Declare an extra namedInput to use the output of the coffee-dealer:build task as a cache input for the coffee-dealer-e2e:e2e task, using the dependentTasksOutputFiles property
• Add a dependency on the build tasks to create a race condition between the coffee-dealer:build tasks that coffee-dealer:serve and coffee-dealer-e2e:e2e depend on.

// apps/coffee-dealer-e2e/project.json
{
  "name": "coffee-dealer-e2e",
  "$schema": "../../node_modules/nx/schemas/project-schema.json",
  "implicitDependencies": ["coffee-dealer"],
  "namedInputs": {
    "coffee-dealer": [
      {
        "dependentTasksOutputFiles": "{workspaceRoot}/dist/apps/coffee-dealer/**/*.js",
        "transitive": false
      }
    ]
  },
  "projectType": "application",
  "targets": {
    "e2e": {
      "executor": "@nx/jest:jest",
      "dependsOn": ["build", "check-port"],
      "inputs": ["^production", "coffee-dealer"],
      "outputs": ["{workspaceRoot}/coverage/{e2eProjectRoot}"],
      "options": {
        "jestConfig": "apps/coffee-dealer-e2e/jest.config.ts",
        "passWithNoTests": true
      },
      "configurations": {
        "ci": {
          "ci": true
        },
        "development": {}
      }
    },
    "check-port": {
      "executor": "nx:run-commands",
      "cache": false,
      "options": {
        "command": "lsof -i:${PORT:-3000} -t"
      }
    },
    "lint": {
      "executor": "@nx/eslint:lint",
      "outputs": ["{options.outputFile}"]
    }
  }
}

Finally, ensure that coffee-dealer-e2e:e2e and coffee-dealer:serve will use the development configuration to use the same build output by setting the defaultConfiguration to development in the serve and build targets of the coffee-dealer project.

// apps/coffee-dealer/project.json
{
  "name": "coffee-dealer",
  "$schema": "../../node_modules/nx/schemas/project-schema.json",
  "sourceRoot": "apps/coffee-dealer/src",
  "projectType": "application",
  "targets": {
    "build": {
      "executor": "@nx/webpack:webpack",
      "outputs": ["{options.outputPath}"],
      "defaultConfiguration": "development",
      "options": {
        "target": "node",
        "compiler": "tsc",
        "transformers": [
          "typia/lib/transform",
          {
            "name": "@nestia/core/lib/transform",
            "options": {
              "validate": "assert",
              "stringify": "assert"
            }
          }
        ],
        "outputPath": "dist/apps/coffee-dealer",
        "main": "apps/coffee-dealer/src/main.ts",
        "tsConfig": "apps/coffee-dealer/tsconfig.app.json",
        "assets": ["apps/coffee-dealer/src/assets"],
        "webpackConfig": "apps/coffee-dealer/webpack.config.js"
      },
      "configurations": {
        "development": {},
        "production": {}
      }
    },
    "serve": {
      "executor": "@nx/js:node",
      "defaultConfiguration": "development",
      "options": {
        "buildTarget": "coffee-dealer:build"
      },
      "configurations": {
        "development": {
          "buildTarget": "coffee-dealer:build:development"
        },
        "production": {
          "buildTarget": "coffee-dealer:build:production"
        }
      }
    }
    // ...
  },
  "tags": ["type:app", "platform:node"]
}

We can now check that the task pipeline is correct :

Note

In this tasks graph, even though the coffee-dealer:serve task is not directly added to the coffee-dealer-e2e:e2e pipeline, they will both produce and consume the same build artifacts. Which is how we implictly tie the coffee-dealer-e2e:e2e task to the coffee-dealer:serve task.

---

We can test the solution with the script test-race-watch.js, which emulates a user's behavior. The goal is to demonstrate that it is possible to ensure that the e2e tests always run against the latest application version.

The script will change an API response from the coffee-dealer app and revert it after a few seconds, triggering a rebuild of the application. The e2e tests will run before and after each rebuild, and we should see the following output:

• e2e tests will fail when the API is changed, and the task won't use the cache
• e2e tests will pass when the API change is reverted, and the tasks will use the cache (when it is populated) for all tasks

// test-race-watch.js
const { exec } = require('node:child_process');
const { assert } = require('node:console');
const { readFile, writeFile } = require('node:fs').promises;
const { join } = require('node:path');
const { PerformanceObserver, performance } = require('node:perf_hooks');
const { once } = require('node:stream');
const { format } = require('node:util');

const coffeeDealerAppServiceTsPath = join(
  __dirname,
  'apps',
  'coffee-dealer',
  'src',
  'app',
  'app.service.ts'
);
const port = 3010;
let counter = 0;
let isOriginalApi = true;

function runE2ETests(suite = '#1') {
  const shouldFail = !isOriginalApi;
  console.warn(
    '➡️ running E2E tests suite %s %s',
    suite,
    shouldFail ? 'should fail' : 'should pass'
  );
  const coffeeDealerE2EProcess = exec('npx nx run coffee-dealer-e2e:e2e', {
    env: { ...process.env, PORT: port },
  });

  coffeeDealerE2EProcess
    .once('spawn', () => {
      performance.mark(format('e2e-start:%s', suite));
      performance.mark(format('e2e-deps-start:%s', suite));
    })
    .once('exit', (code, signal) => {
      console.warn('🏁 test finished with code %s and signal %s', code, signal);
      performance.mark(format('e2e-end:%s', suite));
      performance.measure(
        format('e2e-run:%s', suite),
        format('e2e-start:%s', suite),
        format('e2e-end:%s', suite)
      );
      assert(
        shouldFail ? code !== 0 : code === 0,
        `e2e tests ${suite} should ${shouldFail ? 'fail' : 'pass'}`
      );
    });

  coffeeDealerE2EProcess.stdout.setEncoding('utf8').on('data', (data) => {
    // uncomment to see the output of the e2e tests task
    // console.log(data.trim());
    if (data.includes('Setting up')) {
      performance.mark(format('e2e-deps-end:%s', suite));
      performance.measure(
        format('e2e dependencies tasks:%s', suite),
        format('e2e-deps-start:%s', suite),
        format('e2e-deps-end:%s', suite)
      );
    }
  });
  coffeeDealerE2EProcess.stderr.pipe(process.stderr);
}

async function updateAppService() {
  const data = await readFile(coffeeDealerAppServiceTsPath, 'utf8');
  /**
   * Hello API is the value that should be returned by the API and what the e2e tests expect
   */
  const index = data.search('Hello API');
  let updatedData = '';
  if (index > 0) {
    // in that case e2e tests will fail
    updatedData = data.replaceAll('Hello API', 'Hello World!');
    isOriginalApi = false;
  } else {
    // in that case e2e tests will pass
    updatedData = data.replaceAll('Hello World!', 'Hello API');
    isOriginalApi = true;
  }

  console.warn(
    '🎬 updating API to %s version',
    isOriginalApi ? 'original' : 'new'
  );
  runE2ETests('#1');

  /**
   * should trigger build and reload of cofee-dealer app
   **/
  writeFile(coffeeDealerAppServiceTsPath, updatedData).then(() => {
    counter++;
    console.warn('➡️ updated API #%d', counter);
    runE2ETests('#3');
  });

  runE2ETests('#2');
}

function main() {
  const obs = new PerformanceObserver((items) => {
    console.log(
      'task %s duration : %d ms',
      items.getEntries()[0].name,
      items.getEntries()[0].duration
    );
    performance.clearMarks(items.getEntries()[0].name);
  });
  obs.observe({ type: 'measure' });

  const coffeeDealerServerProcess = exec(
    'npx nx run coffee-dealer:serve --inspect=false',
    {
      env: {
        ...process.env,
        PORT: port,
      },
    }
  );
  // uncomment to see the output of the server
  // coffeeDealerServerProcess.stdout.pipe(process.stdout);
  coffeeDealerServerProcess.stderr.pipe(process.stderr);
  once(coffeeDealerServerProcess, 'exit').then(([code, signal]) =>
    console.log('coffeeDealerServerProcess exit', code, signal)
  );

  /**
   * update API every 3-7 seconds
   */
  setInterval(() => {
    updateAppService();
  }, (Math.floor(Math.random() * 4) + 3) * 1000);
}

main();

coffee-dealer-cli

Link to the application

We can apply the same techniques to the coffee-dealer-cli application by applying similar changes to the project.json file.

Key Takeaways

To conclude our cases, Nx offers many ways to manage implicit internal and external dependencies. We can declare them in the project.json file, use named inputs to cache the output of a task, and use the dependsOn property to create a task pipeline. We can also use the targetDefaults property in the nx.json file to set global rules for the cache and the named inputs.

I hope this article has helped you understand the importance of managing implicit dependencies in your workspace or inspired you to improve it. If you have any questions or feedback, feel free to reach out to me on Twitter or Linkedin.

In the next episode, I plan to explore the usage of nx watch to observe the workspace changes and create a custom tasks pipeline triggered based on the changes.