.png)
When developing a JavaScript package with npm, direct dependencies are defined within the dependencies section of the package.json file. Developers manage these dependencies' versions using semver-compliant version specifications. This allows for precise control, from specifying exact versions to defining ranges that permit the package manager to select compatible versions.
Transitive dependencies, the dependencies of your direct dependencies, present a more complex scenario. Their versions are dictated by the package.json of their immediate parent. Typically, developers defer to the parent library's maintainers. Modifying transitive dependencies carries inherent risks, as even minor changes can introduce subtle, undocumented breaking changes.
However, certain situations necessitate such modifications:
Prior to npm version 8.3.0, controlling transitive dependency versions was challenging. The introduction of the overrides feature aimed to address this.
At first glance, the overrides feature presents a deceptively straightforward appearance:
.png)
{
"overrides": {
"foo": "1.0.0"
}
}
This, ostensibly, instructs npm to enforce version 1.0.0 of foo throughout the dependency tree, regardless of the versions dictated by its parent dependencies. However, the overrides section offers a degree of flexibility that transcends this simple example. Consider:
{
"overrides": {
"bar": {
"foo": "1.0.0"
}
}
}
This configuration targets foo version 1.0.0, but only within the context of bar's dependency tree, not necessarily as a direct dependency. Furthermore, version specifiers can be employed for more nuanced control:
{
"overrides": {
"bar@2.0.0": {
"foo": "1.0.0"
}
}
}
This will override foo's version, but only when it appears within the dependency tree of bar version 2.0.0. The complexity can be compounded:
"overrides": {
"bar@2.0.0": {
"foo": "1.0.0"
},
"bar@2.0.1": {
"foo": "2.0.0"
},
"foo": "3.0.0"
}
As demonstrated, the overrides feature possesses significant semantic power, theoretically enabling granular control over transitive dependency versions. In theory, one could precisely shape their dependency tree. Unfortunately, the implementation was fraught with issues, including inconsistent behavior, problems with updates, and failing to apply nested overrides.
The most significant issue was the inconsistent behavior of the overrides feature. It didn't simply fail with an error or ignore the overrides field. Instead, npm exhibited inconsistencies between successive npm install executions.
Specifically, running npm install twice in a row, without any intervening changes, could result in differing package-lock.json files. The initial run might correctly apply the override, while the subsequent run would silently revert to the original, overridden version, without any warning or error indication.
Given that many CI/CD pipelines run npm install multiple times, it's probable that numerous users of the overrides feature unknowingly deployed the original version instead of the intended, overridden version.
To reproduce this inconsistent behavior, utilize an npm version within the range of 8.3.0 (inclusive) to 11.2.0 (exclusive), and employ the following package.json configuration:
{
"name": "test",
"version": "1.0.0",
"engines": {
"npm": ">=8.3.0"
},
"dependencies": {
"json-server": "0.17.0"
},
"overrides": {
"json-server": {
"package-json": "7.0.0"
}
}
}
After the first run we get:
added 196 packages, and audited 197 packages in 13s
38 packages are looking for funding
run `npm fund` for details
found 0 vulnerabilities
But after the second run we get:
added 20 packages, removed 23 packages, and audited 194 packages in 2s
30 packages are looking for funding
run `npm fund` for details
5 moderate severity vulnerabilities
What’s going on?
Running the npm ls --all flag can reveal more information.
After the first run we get:
`-- json-server@0.17.0 overridden
…
+-- update-notifier@5.1.0
| +-- latest-version@5.1.0
| | `-- package-json@7.0.0 invalid: "^6.3.0" from node_modules/latest-version
| | +-- got@11.8.6
…npm error code ELSPROBLEMS
npm error invalid: package-json@7.0.0 C:\Work\overrides\node_modules\package-json
npm error A complete log of this run can be found in:
Whereas after the second run:
`-- json-server@0.17.0 overridden
…
+-- update-notifier@5.1.0
| +-- latest-version@5.1.0
| | `-- package-json@6.5.0
| | +-- got@9.6.0
…
Upon execution, several observations become immediately apparent:
Seal Security discovered the inconsistency independently in May 2023. The issue had been present since the overrides feature was introduced in December 2021, and was reported in November 2022, but remained unresolved for approximately six months by that time. However, it was classified as a low-priority bug, as the security ramifications were not fully appreciated. It was perceived merely as a flaw within a relatively new, advanced feature.
Upon recognizing the security implications, we promptly understood the severity. Inconsistent npm install results are problematic, but silently reverting to a vulnerable, overridden version is potentially dangerous. Given that security vulnerability mitigation is a primary driver for using overrides, this silent reversion constituted a significant risk.
We engaged with the npm maintainers, and after elucidating the security implications, the issue's priority was elevated to priority zero. We anticipated a resolution within a few months.
However, the situation did not unfold as expected. Several months elapsed with no apparent progress. By November 2023 it became evident that the maintainers lacked the resources to address the complex issue. Consequently, we resolved to develop our own patch for npm.
To grasp the intricacies of this issue, it's essential to understand npm's dependency resolution mechanism.
npm constructs a dependency tree, where nodes represent packages and edges denote dependency relations. A single node may have multiple incoming edges, signifying that various packages depend on it. Each node typically possesses numerous outgoing edges, representing its own dependencies. The project's package.json defines the root node, direct dependencies are its children, and the tree expands from there.
Initially, during tree construction, each node represents a package without a specific version assigned. Multiple edges with differing version specifications may converge on the same node. During resolution — the process of determining the correct version for each node — version conflicts can arise. In such instances, a new node is created for the package, and an existing edge is rerouted to this new node, enabling multiple versions of the same package within the project.
Overrides introduce a late-stage modification to this process. Override requirements, originating from a distinct section of package.json, are not inherently part of nodes or edges. They are attributed to the root node and propagated down the tree. Generally, each edge inherits the override requirements of its source node, and each node inherits the override requirements of its incoming edges. The sole exception occurs when traversing a node that satisfies an override condition, at which point conditional overrides become relevant.
Specifically, given the following overrides configuration, only once traversing bar@2.0.0 does the foo override become relevant.
{
"overrides": {
"bar@2.0.0": {
"foo": "1.0.0"
}
}
}At this juncture, it's natural to contemplate the potential for conflicts arising from multiple override requirements. Consider this scenario:
"overrides": {
"bar@2.0.0": {
"foo": "1.0.0"
},
"foo": "3.0.0"
}
In this case it’s somewhat intuitive that the general rule — foo version 3.0.0 — loses to the more specific rule — foo version 1.0.0 under bar@2.0.0. However, what about this configuration:
"overrides": {
"bar": {
"foo": "1.0.0"
},
"baz": {
"foo": "2.0.0"
},
"foo": "3.0.0"
}
One could deduce that foo version 3.0.0 is the default, with exceptions for foo version 1.0.0 under bar and foo version 2.0.0 under baz. However, if foo is a dependency of both bar and baz, which version should be used? Given the conflict, it's logical to create distinct nodes for foo, adhering to the respective override specifications.
But what if bar is itself a dependency of baz? This edge case raises the question of which version of foo should be used?
For an even more challenging example, let’s consider what happens if both bar and baz depend on an identical version of koo, which in turn depends on foo.
The correct outcome would be to employ different foo versions under each instance of koo. Normally when building the dependency tree, if npm sees two identical versions of the same package npm maps them to the same node, as it’s much more efficient. But if we need different versions of foo then we need an additional copy of koo and all the nodes under it. Npm would be forced to duplicate the entire subtree, only because at some point the override sets might require a different version for the same package. Even if it’s later fixed by a deduplication step, it would create a significant performance penalty for running npm install on any large project with overrides.
All this is to say, that the overrides feature is inherently complex, replete with edge cases, and we haven't even touched upon workspaces, optional and peer dependencies, and other related complications. So bugs are to be expected. But the inconsistency issue did not require some elaborate setup. Even very simple cases proved to be unreliable.
Diving into the code, we quickly found numerous bugs:
Our assessment was that the overrides feature was fundamentally broken. It didn't merely fail in obscure edge cases; it routinely failed in any non-trivial usage scenario. However, this situation did have one silver lining: it made it relatively easy to create numerous test cases, which were crucial for developing and debugging our fix.
The pull request, a complex patch requiring careful review, was submitted. However, due to the npm maintainers' limited availability, it remained unreviewed for nearly a year by October 2024. Even npm 11 was released without the critical fix.
At this point, John-David Dalton from Socket stepped in, engaging with the npm team to re-prioritize the issue. While the maintainers couldn't implement the fix themselves, they authorized John to conduct the code review.
The review was detailed, and by November, progress was visible. However, fixing existing tests and creating new ones proved difficult, as our test cases relied on real-world libraries with complex dependency trees, making mocking impractical.
In December 2024, the npm team confirmed renewed prioritization of the pull request. By February 2025, an npm maintainer began actively addressing and adding missing tests. The final pull request was merged on February 26, 2025, resolving an issue initially reported back on November 13, 2022.
The overrides feature has been significantly improved, but it's important to acknowledge that some core deficiencies remain, which likely require a complete rewrite and a more complex dependency tree resolution process to fully rectify.
While these issues are less prevalent and primarily surface in edge cases, they underscore the need for further refinement to ensure complete adherence to the feature's RFC specifications.
Nevertheless, significant progress has been made. npm install now exhibits consistent behavior across multiple runs, overridden packages are accurately identified, and the fundamental logic underlying the feature is considerably more reliable.
Open source provides significant benefits to developers, with widespread library usage streamlining development. However, the reality of open source maintenance often diverges from the ideal. Many core libraries are maintained by small teams with limited resources. As a result, bugs, including security vulnerabilities, can remain unresolved for extended periods. The fact that two security companies were required to drive a critical fix highlights the gap between perception and reality.
Huge thanks to John-David Dalton from Socket for getting the npm team to pay attention, and for his help with the code review.
And thanks to the npm team for finally getting the fix merged and out there.
Here's a concise timeline:
https://docs.npmjs.com/cli/v11/configuring-npm/package-json
https://github.com/npm/cli/issues/5850 *
https://github.com/npm/cli/issues/5443
https://github.com/npm/cli/issues/5914
https://github.com/npm/cli/pull/7025
https://github.com/npm/cli/pull/8089
https://github.com/npm/rfcs/blob/main/accepted/0036-overrides.md