Resolving Circular Dependency Issues in ES5 Projects

Tonibell — Tue, 03 Sep 2024 10:46:36 +0000

A Guide to Identifying and Fixing Circular Dependencies in JavaScript Projects Using Madge and ESLint.

TL;DR

It is recommended to use ESLint with the rule to check for circular dependencies in your project.
If your build target is ES5, once a module exports constants, it should not import other modules.

Issue Symptoms

When running the project, a constant referenced is output as undefined.

For example: FOO exported from utils.js is imported in index.js and its value is printed as undefined.

// utils.js

// import other modules…
export const FOO = 'foo';

// ...

// index.js

import { FOO } from './utils.js';
// import other modules…

console.log(FOO); // `console.log` outputs `undefined`

// ...

Based on experience, this issue is likely caused by a circular dependency between index.js and utils.js.

The next step is to identify the circular dependency path between the two modules to verify the hypothesis.

Finding Circular Dependencies

Install Madge Tool

There are many tools available in the community for finding circular dependencies. Here, we use Madge as an example.

Madge is a developer tool for generating a visual graph of your module dependencies, finding circular dependencies, and providing other useful information.

Step 1: Configure Madge

// madge.js
const madge = require("madge");
const path = require("path");
const fs = require("fs");
madge("./index.ts", {
  tsConfig: {
    compilerOptions: {
      paths: {
        // specify path aliases if using any
      },
    },
  },
})
  .then((res) => res.circular())
  .then((circular) => {
    if (circular.length > 0) {
      console.log("Found circular dependencies: ", circular);
      // save the result into a file
      const outputPath = path.join(__dirname, "circular-dependencies.json");
      fs.writeFileSync(outputPath, JSON.stringify(circular, null, 2), "utf-8");
      console.log(`Saved to ${outputPath}`);
    } else {
      console.log("No circular dependencies found.");
    }
  })
  .catch((error) => {
    console.error(error);
  });

Step 2: Run the Script

node madge.js

After running the script, a 2D array is obtained.

The 2D array stores all the circular dependencies in the project. Each sub-array represents a specific circular dependency path: file at index n references the file at index n + 1, and the last file references the first file, forming a circular dependency.

It's important to note that Madge can only return direct circular dependencies. If two files form an indirect circular dependency through a third file, it will not be included in Madge's output.

Based on the actual project situation, Madge output a result file with over 6,000 lines. The result file shows that the suspected circular dependency between the two files is not directly referenced. Finding the indirect dependency between the two target files was like searching for a needle in a haystack.

Writing a Script to Find Indirect Circular Dependencies

Next, I asked ChatGPT to help write a script to find direct or indirect circular dependency paths between two target files based on the result file.

/**
 * Check if there is a direct or indirect circular dependency between two files
 * @param {Array<string>} targetFiles Array containing two file paths
 * @param {Array<Array<string>>} references 2D array representing all file dependencies in the project
 * @returns {Array<string>} Array representing the circular dependency path between the two target files
 */
function checkCircularDependency(targetFiles, references) {
  // Build graph
  const graph = buildGraph(references); // Store visited nodes to avoid revisiting
  let visited = new Set(); // Store the current path to detect circular dependencies
  let pathStack = [];
  // Depth-First Search
  function dfs(node, target, visited, pathStack) {
    if (node === target) {
      // Found target, return path
      pathStack.push(node);
      return true;
    }
    if (visited.has(node)) {
      return false;
    }
    visited.add(node);
    pathStack.push(node);
    const neighbors = graph[node] || [];
    for (let neighbor of neighbors) {
      if (dfs(neighbor, target, visited, pathStack)) {
        return true;
      }
    }
    pathStack.pop();
    return false;
  }
  // Build graph
  function buildGraph(references) {
    const graph = {};
    references.forEach((ref) => {
      for (let i = 0; i < ref.length; i++) {
        const from = ref[i];
        const to = ref[(i + 1) % ref.length]; // Circular reference to the first element
        if (!graph[from]) {
          graph[from] = [];
        }
        graph[from].push(to);
      }
    });
    return graph;
  }
  // Try to find the path from the first file to the second file
  if (dfs(targetFiles[0], targetFiles[1], new Set(), [])) {
    // Clear visited records and path stack, try to find the path from the second file back to the first file
    visited = new Set();
    pathStack = [];
    if (dfs(targetFiles[1], targetFiles[0], visited, pathStack)) {
      return pathStack;
    }
  }
  // If no circular dependency is found, return an empty array
  return [];
}
// Example usage
const targetFiles = [
  "scene/home/controller/home-controller/grocery-entry.ts",
  "../../request/api/home.ts",
];
const references = require("./circular-dependencies");
const circularPath = checkCircularDependency(targetFiles, references);
console.log(circularPath);

Using the 2D array output from Madge as the script input, the result indicated that there was indeed a circular dependency between index.js and utils.js, consisting of a chain involving 26 files.

Root Cause

Before solving the problem, we need to understand the root cause: Why does circular dependency cause the referenced constant to be undefined?

To simulate and simplify the problem, let's assume the circular dependency chain is as follows:

index.js → component-entry.js → request.js → utils.js → component-entry.js

Since the project code is ultimately bundled by Webpack and compiled into ES5 code using Babel, we need to look at the structure of the bundled code.

Example of Webpack Bundled Code

(() => {
  "use strict";

  var e,
    __modules__ = {
      /* ===== component-entry.js starts ==== */
      148: (_, exports, __webpack_require__) => {
        // [2] define the getter of `exports` properties of `component-entry.js`
        __webpack_require__.d(exports, { Cc: () => r, bg: () => c });
        // [3] import `request.js`
        var t = __webpack_require__(595);
        // [9]
        var r = function () {
            return (
              console.log("A function inside component-entry.js run, ", c)
            );
          },
          c = "An constants which comes from component-entry.js";
      },
      /* ===== component-entry.js ends ==== */

      /* ===== request.js starts ==== */
      595: (_, exports, __webpack_require__) => {
        // [4] import `utils.js`
        var t = __webpack_require__(51);
        // [8]
        console.log("request.js run, two constants from utils.js are: ", t.R, ", and ", t.b);
      },
      /* ===== request.js ends ==== */

      /* ===== utils.js starts ==== */
      51: (_, exports, __webpack_require__) => {
        // [5] define the getter of `exports` properties of `utils.js`
        __webpack_require__.d(exports, { R: () => r, b: () => t.bg });
        // [6] import `component-entry.js`, `component-entry.js` is already in `__webpack_module_cache__`
        // so `__webpack_require__(148)` will return the `exports` object of `component-entry.js` immediately
        var t = __webpack_require__(148);
        var r = 1001;
        // [7] print the value of `bg` exported by `component-entry.js`
        console.log('utils.js,', t.bg); // output: 'utils, undefined'
      },
      /* ===== utils.js starts ==== */
    },

    __webpack_module_cache__ = {};

  function __webpack_require__(moduleId) {
    var e = __webpack_module_cache__[moduleId];

    if (void 0 !== e) return e.exports;

    var c = (__webpack_module_cache__[moduleId] = { exports: {} });

    return __modules__[moduleId](c, c.exports, __webpack_require__), c.exports;
  }

  // Adds properties from the second object to the first object
  __webpack_require__.d = (o, e) => {
    for (var n in e)
      Object.prototype.hasOwnProperty.call(e, n) &&
        !Object.prototype.hasOwnProperty.call(o, n) &&
        Object.defineProperty(o, n, { enumerable: !0, get: e[n] });
  },

  // [0]
  // ======== index.js starts ========
  // [1] import `component-entry.js`
  (e = __webpack_require__(148/* (148 is the internal module id of `component-entry.js`) */)),
  // [10] run `Cc` function exported by `component-entry.js`
  (0, e.Cc)();
  // ======== index.js ends ========
})();

In the example, [number] indicates the execution order of the code.

Simplified version:

function lazyCopy (target, source) {
  for (var ele in source) {
    if (Object.prototype.hasOwnProperty.call(source, ele)
      && !Object.prototype.hasOwnProperty.call(target, ele)
    ) {
      Object.defineProperty(target, ele, { enumerable: true, get: source[ele] });
    }
  }
}

// Assuming module1 is the module being cyclically referenced (module1 is a webpack internal module, actually representing a file)
var module1 = {};
module1.exports = {};
lazyCopy(module1.exports, { foo: () => exportEleOfA, print: () => print, printButThrowError: () => printButThrowError });
// module1 is initially imported at this point

// Assume the intermediate process is omitted: module1 references other modules, and those modules reference module1

// When module1 is imported a second time and its `foo` variable is used, it is equivalent to executing:
console.log('Output during circular reference (undefined is expected): ', module1.exports.foo); // Output `undefined`

// Call `print` function, which can be executed normally due to function scope hoisting
module1.exports.print(); // 'print function executed'

// Call `printButThrowError` function, which will throw an error due to the way it is defined
try {
  module1.exports.printButThrowError();
} catch (e) {
  console.error('Expected error: ', e); // Error: module1.exports.printButThrowError is not a function
}

// Assume the intermediate process is omitted: all modules referenced by module1 are executed

// module1 returns to its own code and continues executing its remaining logic
var exportEleOfA = 'foo';
function print () {
  console.log('print function executed');
}
var printButThrowError = function () {
  console.log('printButThrowError function executed');
}

console.log('Normal output: ', module1.exports.foo); // 'foo'
module1.exports.print(); // 'print function executed'
module1.exports.printButThrowError(); // 'printButThrowError function executed'

Webpack Module Bundling Process

During the AST analysis phase, Webpack looks for ES6 import and export statements. If a file contains these statements, Webpack marks the module as "harmony" type and performs corresponding code transformation for exports:

https://github.com/webpack/webpack/blob/c586c7b1e027e1d252d68b4372f08a9bce40d96c/lib/dependencies/HarmonyExportInitFragment.js#L161

https://github.com/webpack/webpack/blob/c586c7b1e027e1d252d68b4372f08a9bce40d96c/lib/RuntimeTemplate.js#L164

Summary of Root Cause

Issue symptom: A module imports a constant, but its actual value is undefined when running.
Conditions for the issue to occur:
- The project is bundled by Webpack and compiled into ES5 code (not tested with other bundlers).
- Module A defines a constant foo and exports it, and this module has circular dependencies with other modules.
- Module B imports foo from Module A and runs during the module initialization process.
- Module A and Module B have a circular dependency.
Root causes:
- In Webpack's module system, when a module is first referenced, Webpack initializes its exports using property getters and stores it in a cache object. When the module is referenced again, it directly returns the exports from the cache object.
- let variables and const constants are compiled into var declarations, causing variable hoisting issues. When used before their actual definition, they return undefined but do not throw an error.
- Function declarations are hoisted, allowing them to be called normally.
- Arrow functions are compiled into var foo = function () {}; and function expressions do not have function scope hoisting. Therefore, they throw an error when run instead of returning undefined.

How to Avoid

ESLint

We can use ESLint to check for circular dependencies in the project. Install the eslint-plugin-import plugin and configure it:

// babel.config.js

import importPlugin from 'eslint-plugin-import';

export default [
  {
    plugins: {
      import: importPlugin,
    },
    rules: {
      'import/no-cycle': ['error', { maxDepth: Infinity }],
    },
    languageOptions: {
      "parserOptions": {
        "ecmaVersion": 6, // or use 6 for ES6
        "sourceType": "module"
      },
    },
    settings: {
      // Need this to let 'import/no-cycle' to work
      // reference: https://github.com/import-js/eslint-plugin-import/issues/2556#issuecomment-1419518561
      "import/parsers": {
        espree: [".js", ".cjs", ".mjs", ".jsx"],
      }
    },
  },
];

DEV Community: Tonibell