During a frontend development project, a new npm package was added, and conflicts were encountered when merging the lockfile.
Lockfile refers to the lock file of package management tools, such as
package-lock.json,yarn.lock,pnpm-lock.yaml.
Manually resolving conflicts can be highly inefficient and error-prone. Here are several commonly used solutions:
- Delete the lockfile and reinstall the dependencies later.
- Reset the lockfile to that of one branch and reinstall the dependencies later.
- Run the dependency installation command to leverage the built-in mechanism of the package management tool to resolve lockfile conflicts.
Option 1 will result in the loss of lock records and is generally not preferred.
Are options 2 and 3 feasible? What are the considerations? This article will discuss these questions and provide the best practices at the end.
If you prefer to skip the details, you can scroll to the end directly to see the "Best Practices."
Before we dive into these questions, let's first understand the situations in which lockfile merge conflicts can occur.
Reasons for Lockfile Merge Conflicts
The reason for conflicts during a Git merge is when two branches make modifications to the same region of a single file. If the modifications are in different regions, Git will attempt an automatic merge (auto-merge) to resolve the conflict.
If you're not familiar with resolving Git merge conflicts, you can refer to the article How to Resolve Merge Conflicts in Git – A Practical Guide with Examples.
The key to merge conflicts is changes in the same region. Let's take the dependency configuration in package.json as an example. The first two examples below illustrate a conflicting scenario and a non-conflicting scenario, respectively.
For lockfiles, changes in the same region typically occur in the following two situations:
- Incompatible merging due to differences in the structure of the lockfile caused by inconsistent versions of the package management tool. To address this issue, it is recommended to maintain consistency in the package management tool's version. You can refer to my other article: Why is it so hard to maintain consistent frontend dependencies? - 掘金.
- Both branches have made changes to the dependency configuration in
package.jsonand modified the same region in the lockfile.
When does the same region of the lockfile get modified?
This is a complex issue as different package management tools and their versions have different generation strategies. It is difficult for developers to avoid conflicts by adjusting the writing style of package.json. Therefore, it is not necessary to focus too much on this aspect. The important thing is to know how to resolve the issue when it occurs.
Next, we will discuss two commonly used resolution strategies.
Analysis of the Solutions: Resetting Branch Lockfile
TL;DR: Resetting the branch lockfile means restoring the lockfile to the version from the target branch or the current branch, which also means losing some lock records from a branch and may lead to errors. This issue is challenging to resolve completely and can only be mitigated by improving the development process and performing necessary manual reviews.
Resetting the branch lockfile refers to "merging by using the lockfile from the target branch or the current branch" and then re-executing the dependency installation command to update the lockfile.
Three Resetting Solutions
The following three solutions can be used to easily reset the branch lockfile:
- Ignore the conflict prompt and stash the lockfile changes, then discard those changes, indicating the continuation of the lockfile from the current branch.
- Execute the
git checkout --ours "*lock*"orgit checkout --theirs "*lock*"command to automatically resolve lockfile conflicts based on the current branch or target branch. -
Implement lockfile conflict resolution automation based on Git configuration. This requires two essential steps, both of which are necessary:
- Add a
.gitattributesfile to configure the merge strategy. Example: ```
# .gitattributes
# When there's a conflict in pnpm-lock.yaml, use the lockfile from the current branch
pnpm-lock.yaml merge=ours - Add a
2. Execute the `git config merge.ours.driver true` command to enable the merge driver configuration (if using the `theirs` merge strategy, the command should be changed to `merge.theirs.driver`).
3. Reference documentation:
1. [Have Git Select Local Version On Merge Conflict on a Specific File?](http://stackoverflow.com/a/930495/958481)
2. [Merge Strategies](https://git-scm.com/book/en/v2/Customizing-Git-Git-Attributes#Merge-Strategies:~:text=further%20development%20work.-,Merge%20Strategies,-You%20can%20also)
## Existing Issues
Regardless of whether the lockfile is reset based on the **current branch** or the **target branch** and the subsequent update of dependencies, it **means losing some lock records and may lead to errors**.
Here's an example encountered in a real business scenario:
1. From the main branch, create a development branch called `feat1` with a new dependency `A^1.0.0`, and the installed version is `1.0.0`.
2. During the development of `feat1`, another development branch, `feat2`, is merged into the main branch and adds a dependency `B^2.0.0`, with the installed version being `2.0.0`.
3. After completing the development in `feat1` and preparing to merge it into the main branch, a lockfile conflict is discovered.
4. Using the aforementioned **"Resetting Branch Lockfile"** solution, the dependencies are reinstalled.
5. At this time, the community releases versions `A@1.0.1` and `B@2.0.1` with *`BREAKING CHANGE`*. Since the **"Resetting Branch Lockfile"** solution ignores lock records added by `feat1` or the main branch (`feat2`), it installs the newer versions `A@1.0.1` or `B@2.0.1`.
6. `feat1` is directly merged into the main branch, causing errors in the production code.
## Solution
Some people may suggest **using fixed versions when installing dependencies** instead of using version ranges. For example, using `"A": "1.0.0"` and `"B": "2.0.0"` as dependencies instead of using version range notation (`^`).
{
"dependencies": {
"A": "1.0.0",
"B": "2.0.0",
}
}
Firstly, for **application projects**, using fixed versions is feasible. However, for **library projects**, it is not recommended to use fixed versions for the following reasons:
- The dependencies of the library project cannot be efficiently reused by dependent application projects (e.g., `^1.0.0` and `^1.1.0` can be merged into `^1.1.0`).
- In the event of security vulnerabilities in the **indirect dependencies** of the library project, they cannot be directly fixed by reinstalling the dependencies.
Furthermore, locking the versions of direct dependencies is not a complete solution as losing the lockfile can still lead to the automatic upgrade of indirectly dependent dependencies, which can result in *`BREAKING CHANGE`*.
Therefore, once this solution is chosen, either **trust that other dependencies will not have issues (leave it to chance)** or perform **necessary manual lockfile reviews** and ensure it through **a well-defined development process**.
- **Well-defined development process**: During the development stage, promptly merge or rebase code from the main branch, resolve conflicts ahead of time rather than waiting until just before testing and deployment. Before merging after testing, if code conflicts are found, resolve them and retest the relevant code to ensure it works as expected (if project importance and manpower allow).
- **Necessary manual lockfile reviews**: Focus only on changes to **direct dependencies** (e.g., the `specifier` and `version` sections in the `pnpm-lock.yaml` file). It is less likely to encounter automatic upgrade errors for **indirect dependencies** introduced by direct dependencies (since multiple projects would be affected if errors occurred) and the review cost would be too high. Trusting the community or performing "change testing" is a viable option.
## Summary of Solutions
1. Resetting the branch lockfile means restoring the lockfile to the version from the **target branch** or the **current branch**.
2. There are three solutions available to facilitate the resetting of the branch lockfile.
3. Resetting the branch lockfile leads to the loss of lock records from one branch, which can result in errors.
4. Locking the versions of direct dependencies is not the ultimate solution and can lead to other issues.
5. Rely on a well-defined development process, perform conflict testing promptly, and conduct sufficient manual reviews.
# Analysis of the Solution: Package Manager's Built-in Mechanism
> **TL;DR**: The conflicting lockfile can be resolved by parsing the conflicting file into different versions of lock objects and then merging those lock objects. Each package manager has its own merge strategy, with pnpm being the most effective overall. However, regardless of the approach, some data will be lost, and it cannot guarantee 100% error-free merging.
When a lockfile merge conflict occurs, the mainstream package managers support running dependency installation commands (`npm install/yarn/pnpm install`) to automatically resolve the conflict.
- npm: [Resolving lockfile conflicts](https://docs.npmjs.com/cli/v6/configuring-npm/package-locks#resolving-lockfile-conflicts), supported from v5.7 onwards
- yarn: [Auto-merging of lockfiles](https://engineering.fb.com/2017/09/07/web/announcing-yarn-1-0/#:~:text=Auto%2Dmerging%20of%20lockfiles), supported from v1.0 onwards
- pnpm: [Merge conflicts](https://pnpm.io/git#merge-conflicts), supported from v5.11 onwards
> It can be assumed that most users are using versions of the package managers that support this feature.
So, how do these package managers resolve conflicts?
## npm's Conflict Resolution Strategy
I have analyzed this issue in another article, "[A Discussion on the Algorithm for Resolving Merge Conflicts in package-lock.json](https://juejin.cn/post/7251895470548697143)".
In general, the strategy is to **base the resolution on the target branch (`theirs`) and apply the changes from the current branch (`ours`)**.
For example, when merging a `development branch` into the `main branch` (`git merge feat-branch`), `theirs` refers to the `development branch`, and `ours` refers to the `main branch`. In this case, the lock records will be based on the `development branch` and the changes from the `main branch` will be applied.
In other words, **if both branches update the version of the same module, the version from the main branch (`ours`) will be used, which may lead to errors in rare cases**.
The suggested solution in the article is to reduce the impact through processes and retesting, as explained later.
## yarn's Conflict Resolution Strategy
The PR for [Auto detect and merge lockfile conflicts](https://github.com/yarnpkg/yarn/pull/3544) provides insight into the latest implementation in [/src/lockfile/parse.js](https://github.com/yarnpkg/yarn/blob/master/src/lockfile/parse.js#L334).
In fact, understanding just one line of code is sufficient:
Object.assign({}, parse(variants[0], fileLoc), parse(variants[1], fileLoc));
This **shallow merge** combines the YAML objects from both branches, and when the same property exists in both objects, the value from the target branch (`theirs`) will be used.
In other words, when conflicting versions of the same dependency are encountered, **the version from the target branch is used**. This is the opposite of npm's strategy, but the issues and solutions are similar.
## pnpm's Conflict Resolution Strategy
The conflict resolution algorithm for pnpm is maintained by the [@pnpm/merge-lockfile-changes](https://github.com/pnpm/pnpm/tree/main/lockfile/merge-lockfile-changes) project.
The implementation involves breaking down the conflicting parts into the contents from the target branch (`theirs`) and the current branch (`ours`), and then merging them. However, unlike yarn's straightforward shallow merge, pnpm performs a **deep merge** (the lockfile structure only has two levels) and selects the version with the higher version number when there is a version conflict.
Here's a demo test:
```js
const { mergeLockfileChanges } = require("@pnpm/merge-lockfile-changes");
const simpleLockfile = {
importers: {},
lockfileVersion: 5.2,
};
const mergedLockfile = mergeLockfileChanges(
{
...simpleLockfile,
packages: {
".": {
version: "1.1.0",
dependencies: {
foo: "1.2.0",
bar: "3.0.0_qar@1.0.0",
zoo: "4.0.0_qar@1.0.0",
ktv: "5.0.0"
},
},
},
},
{
...simpleLockfile,
packages: {
".": {
version: "1.2.0",
dependencies: {
foo: "1.1.0",
bar: "4.0.0_qar@1.0.0",
zoo: "3.0.0_qar@1.0.0",
pua: "5.0.0"
},
},
},
}
);
console.log(JSON.stringify(mergedLockfile, null, 2));
The output will be:
{
"importers": {},
"lockfileVersion": 5.2,
"packages": {
".": {
"version": "1.2.0",
"dependencies": {
"foo": "1.2.0",
"bar": "4.0.0_qar@1.0.0",
"zoo": "4.0.0_qar@1.0.0",
"ktv": "5.0.0",
"pua": "5.0.0"
}
}
}
}
In pnpm's strategy, the updated version is chosen. If there is a problem, it implies that the new version has introduced a BREAKING CHANGE. However, the probability of encountering this issue is lower compared to selecting an older version and facing a BREAKING CHANGE.
Currently, the pnpm team is continuously improving the lockfile solution to reduce conflicts, including:
- Using different lock files for different branches: https://github.com/pnpm/pnpm/pull/4475
- Changing the lockfile format to reduce the number of conflicts: https://github.com/pnpm/rfcs/pull/1
Additionally, pnpm provides a resolution-mode configuration where users can decide the dependency version selection strategy during installation: lowest (default), highest, or time-based (related to the last direct dependency).
Summary of the Solutions
All three solutions involve merging the lockfile, but they have different merge strategies:
- npm: Deep merge based on the current branch (
ours). - yarn: Shallow merge based on the target branch (
theirs). - pnpm: Deep merge based on the version number, selecting the higher version.
The pnpm solution has the smallest probability of encountering issues, but it is not without its potential problems, as stated in the official documentation:
It is recommended to inspect the changes before submitting, as we can't guarantee that pnpm will choose the correct head—it builds most of the updated lockfile, which is ideal in most cases.
In conclusion, while the pnpm solution has the lowest probability of encountering issues, it is not guaranteed to be error-free. As shown in the diagram above, some issues may still arise.
Best Practices
The built-in mechanisms provided by package managers result in fewer lost lockfile records and fewer issues compared to the approach of resetting the branch's lockfile. Based on the analysis above, we can summarize the best practices as follows:
- Avoid manually resolving conflicts as it can lead to syntax errors.
- Utilize the conflict resolution mechanisms provided by the package manager whenever possible.
- If you have the flexibility to choose a package manager, consider using
pnpm.
Additionally, it's important to note that while the package manager solutions are generally effective, extreme scenarios may still result in issues. If the project is of significant value, it is recommended to perform manual reviews and conduct regression testing. You can refer to the "Analysis of the Solution: Resetting Branch Lockfile" section for specific guidelines on how to approach these situations.
Conclusion
This article provided a comprehensive analysis of common approaches to resolving lockfile conflicts and presented the best practices. It is essential to broaden our perspectives and elevate our understanding of best practices at the software level, which can then be shared with the community.
Finally, if you found this article helpful, please consider giving it a thumbs up, bookmarking it, and sharing it with others. 🍻
Top comments (2)
welcome to discuss / 欢迎讨论
I put pnpm-lock.yaml into .gitignore, so there's never a conflict.
I often build libraries rather than applications, so that I should ensure my package.json contains the proper version constraints, because the dependents of my libraries would not consider what's in my lockfiles.