DEV Community

Cover image for TypeScript Frustration TSFR-001 : Yarn 4 / PnP and dependency resolution
Davy De Waele
Davy De Waele

Posted on

TypeScript Frustration TSFR-001 : Yarn 4 / PnP and dependency resolution

I’ve decided to write-up a weekly report of all the frustrating things I encounter with the JavaScript / TypeScript eco-system during my development adventures.

I’m not by all means a professional JavaScript developer, but I have done several TypeScript projects over the years. And what I keep realizing is I am still lacking some of that core knowledge and experience, as I am sure is also the case for the vast majority of JavaScript developers. Combine that with the fast moving pace of JavaScript frameworks / tooling-chains and you can set yourself up for disaster.

I will try to describe the issue as best as I can and when possible also describe the solution.

I think it's a good way to get a more fundamental understanding on how things work and hopefully it will help others.

So …. without further ado, here’s the frustration of the day :

*TS2307: Cannot find module x or its corresponding type declarations when using Yarn4 and PnP
*

Funny typescript bug image

The issue
Using Yarn 4 as a package manager with PnP (Plug And Play) enabled (the default), I am able to run my typescript code via tsx , but it fails during compilation , and my code editor is also showing errors. (TS2307: Cannot find module .. or its corresponding type declarations.)

The context
Every JavaScript developer is probably familiar with npm and node_modules. When Yarn (classic) came along it continued using node_modules but a new iteration of Yarn v2 (called berry) moved away from that and started using PnP (Plug And Play) as the new way of resolving nodejs dependencies

The solution (for those that are in a hurry)
It appears that default yarn 4 that is used on my system (4.3.1) has an issue resolving dependencies when using PnP. It attempts to look for dependencies in node_modules despite the fact that PnP is used. This issue was present up until 4.5.1 and appears to have been fixed in 4.5.2 and upwards. Specifying a more recent version of the package manager in package.json fixed the issue.

The project

I have a project with the following typescript source. It features a single import from the @langchain/core module

import { ChatPromptTemplate } from "@langchain/core/prompts";

const prompt = ChatPromptTemplate.fromTemplate(
    `Answer the following question to the best of your ability:\n{question}`
);

console.log(await prompt.format({ question: "What is the answer to life, the universe, and everything?" }));
Enter fullscreen mode Exit fullscreen mode

The yarn package manager

I had bootstrapped the project with yarn (notice how it defaults to 4.3.1 — more on that later).

yarn init -y
➤ YN0000: · Yarn 4.3.1
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: └ Completed
➤ YN0000: · Done in 0s 20ms
The @langchain/core dependency
Enter fullscreen mode Exit fullscreen mode

I have installed the @langchain/core dependency with yarn

 yarn add @langchain/core
➤ YN0000: ┌ Resolution step
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0013: │ @langchain/core@npm:0.3.66 can't be found in the cache and will be fetched from the remote registry
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0000: └ Completed
➤ YN0000: Done with warnings in 0s 215ms
Enter fullscreen mode Exit fullscreen mode

I also added the tsx and typescript dependencies

yarn add -D typescript
yarn add -D tsx
Enter fullscreen mode Exit fullscreen mode

My package.json looks fine

{
  "name": "myproject",
  "packageManager": "yarn@4.3.1",
  "type": "module",
  "scripts": {
    "tsx": "tsx"
  },
  "dependencies": {
    "@langchain/core": "^0.3.66"
  },
  "devDependencies": {
    "tsx": "^4.20.3",
    "typescript": "^5.9.2"
  }
}
Enter fullscreen mode Exit fullscreen mode

Running via tsx / IDE

I can run the TypeScript file just fine from the command line using tsx

yarn run tsx src/index.ts
Human: Answer the following question to the best of your ability:
What is the answer to life, the universe, and everything?
Enter fullscreen mode Exit fullscreen mode

I can run it fine from my IDE

Run Debug Config

Console output

TypeScript errors in IDE and TSC

But when I look at the sources in my IDE it keeps saying

TS2307: Cannot find module @langchain/core/prompts or its corresponding type declarations.
Enter fullscreen mode Exit fullscreen mode

This also doesn’t seem to be a WebStorm issue as issuing the tsc command on the command line results in a similar error

yarn tsc
src/index.ts:1:36 - error TS2307: Cannot find module '@langchain/core/prompts' or its corresponding type declarations.
Enter fullscreen mode Exit fullscreen mode

So why is that ?

It is weird as you can see that it has picked up the @langchain/core as an external dependency, and it should be able to resolve the @langchain/core/prompts import

And if I do a reveal in finder I can see that it resolves the dependency from the following file

~/.yarn/berry/cache/@langchain-core-npm-0.3.66-d2b05db5ca-10c0.zip

and indeed inside that zip file we have the correct path

The issue ?

So it appears that tsc is not working when 4.3.1 is used as the package manager

base ❯ yarn install
➤ YN0000: · Yarn 4.3.1
➤ YN0000: ┌ Resolution step
➤ YN0085: │ + typescript@patch:typescript@npm%3A5.9.2#optional!builtin<compat/typescript>::version=5.9.2&hash=379a07
➤ YN0085: │ - typescript@patch:typescript@npm%3A5.9.2#optional!builtin<compat/typescript>::version=5.9.2&hash=cef18b
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0000: └ Completed
➤ YN0000: · Done with warnings in 0s 146ms

~/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/myproject master*
base ❯ yarn tsc
src/index.ts:1:36 - error TS2307: Cannot find module '@langchain/core/prompts' or its corresponding type declarations.

1 import { ChatPromptTemplate } from "@langchain/core/prompts";
                                     ~~~~~~~~~~~~~~~~~~~~~~~~~


Found 1 error in src/index.ts:1
Enter fullscreen mode Exit fullscreen mode

U*pgrading to 4.5.2 fixes the issue*

base ❯ yarn install
➤ YN0000: · Yarn 4.5.2
➤ YN0000: ┌ Resolution step
➤ YN0085: │ + typescript@patch:typescript@npm%3A5.9.2#optional!builtin<compat/typescript>::version=5.9.2&hash=5786d5
➤ YN0085: │ - typescript@patch:typescript@npm%3A5.9.2#optional!builtin<compat/typescript>::version=5.9.2&hash=379a07
➤ YN0000: └ Completed
➤ YN0000: ┌ Fetch step
➤ YN0000: └ Completed
➤ YN0000: ┌ Link step
➤ YN0000: │ ESM support for PnP uses the experimental loader API and is therefore experimental
➤ YN0000: └ Completed
➤ YN0000: · Done with warnings in 0s 148ms

~/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/myproject master*
base ❯ yarn tsc

WORKS
Enter fullscreen mode Exit fullscreen mode

When doing a yarn tsc — traceResolution you can indeed see that with 4.5.2 it is resolving the dependency via the berry cache (PnP)

base ❯ yarn tsc --traceResolution
======== Resolving module '@langchain/core/prompts' from '/Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal2/myproject/src/index.ts'. ========
Explicitly specified module resolution kind: 'Node10'.
Loading module '@langchain/core/prompts' from 'node_modules' folder, target file types: TypeScript, Declaration.
Searching all ancestor node_modules directories for preferred extensions: TypeScript, Declaration.
Found 'package.json' at '/Users/davydewaele/.yarn/berry/cache/@langchain-core-npm-0.3.66-d2b05db5ca-10c0.zip/node_modules/@langchain/core/package.json'.
'package.json' does not have a 'typesVersions' field.
File '/Users/davydewaele/.yarn/berry/cache/@langchain-core-npm-0.3.66-d2b05db5ca-10c0.zip/node_modules/@langchain/core/prompts.ts' does not exist.
File '/Users/davydewaele/.yarn/berry/cache/@langchain-core-npm-0.3.66-d2b05db5ca-10c0.zip/node_modules/@langchain/core/prompts.tsx' does not exist.
File '/Users/davydewaele/.yarn/berry/cache/@langchain-core-npm-0.3.66-d2b05db5ca-10c0.zip/node_modules/@langchain/core/prompts.d.ts' exists - use it as a name resolution result.
'package.json' does not have a 'peerDependencies' field.
Resolving real path for '/Users/davydewaele/.yarn/berry/cache/@langchain-core-npm-0.3.66-d2b05db5ca-10c0.zip/node_modules/@langchain/core/prompts.d.ts', result '/Users/davydewaele/.yarn/berry/cache/@langchain-core-npm-0.3.66-d2b05db5ca-10c0.zip/node_modules/@langchain/core/prompts.d.ts'.
======== Module name '@langchain/core/prompts' was successfully resolved to '/Users/davydewaele/.yarn/berry/cache/@langchain-core-npm-0.3.66-d2b05db5ca-10c0.zip/node_modules/@langchain/core/prompts.d.ts' with Package ID '@langchain/core/prompts.d.ts@0.3.66'. ========
Enter fullscreen mode Exit fullscreen mode

When doing a yarn tsc — traceResolution you can indeed see that with 4.5.1 it is resolving trying to resolve the dependency via node_modules

base ❯ yarn tsc --traceResolution
======== Resolving module '@langchain/core/prompts' from '/Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/myproject/src/index.ts'. ========
Explicitly specified module resolution kind: 'Node10'.
Loading module '@langchain/core/prompts' from 'node_modules' folder, target file types: TypeScript, Declaration.
Searching all ancestor node_modules directories for preferred extensions: TypeScript, Declaration.
Directory '/Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/myproject/src/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'langchain__core/prompts'
Directory '/Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/myproject/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'langchain__core/prompts'
Directory '/Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'langchain__core/prompts'
Directory '/Users/davydewaele/Projects/AgenticAI/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'langchain__core/prompts'
Directory '/Users/davydewaele/Projects/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'langchain__core/prompts'
Directory '/Users/davydewaele/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'langchain__core/prompts'
Directory '/Users/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'langchain__core/prompts'
Directory '/node_modules' does not exist, skipping all lookups in it.
Scoped package detected, looking in 'langchain__core/prompts'
Loading module '@langchain/core/prompts' from 'node_modules' folder, target file types: JavaScript, JSON.
Searching all ancestor node_modules directories for fallback extensions: JavaScript, JSON.
Directory '/Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/myproject/src/node_modules' does not exist, skipping all lookups in it.
Directory '/Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/myproject/node_modules' does not exist, skipping all lookups in it.
Directory '/Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/node_modules' does not exist, skipping all lookups in it.
Directory '/Users/davydewaele/Projects/AgenticAI/node_modules' does not exist, skipping all lookups in it.
Directory '/Users/davydewaele/Projects/node_modules' does not exist, skipping all lookups in it.
Directory '/Users/davydewaele/node_modules' does not exist, skipping all lookups in it.
Directory '/Users/node_modules' does not exist, skipping all lookups in it.
Directory '/node_modules' does not exist, skipping all lookups in it.
======== Module name '@langchain/core/prompts' was not resolved. ========
Enter fullscreen mode Exit fullscreen mode

By using PnP you also need to be aware that the out of the box node will not be able to resolve these modules. If you attempt to launch your compiled javascript using node it will fail

base ❯ node index.js
node:internal/modules/esm/resolve:854
  throw new ERR_MODULE_NOT_FOUND(packageName, fileURLToPath(base), null);
        ^

Error [ERR_MODULE_NOT_FOUND]: Cannot find package '@langchain/core' imported from /Users/davydewaele/Projects/AgenticAI/typescript-debug-langchainjs-yarn-portal3/myproject/dist/index.js
    at packageResolve (node:internal/modules/esm/resolve:854:9)
    at moduleResolve (node:internal/modules/esm/resolve:927:18)
    at defaultResolve (node:internal/modules/esm/resolve:1169:11)
    at ModuleLoader.defaultResolve (node:internal/modules/esm/loader:540:12)
    at ModuleLoader.resolve (node:internal/modules/esm/loader:509:25)
    at ModuleLoader.getModuleJob (node:internal/modules/esm/loader:239:38)
    at ModuleWrap.<anonymous> (node:internal/modules/esm/module_job:96:40)
    at link (node:internal/modules/esm/module_job:95:36) {
  code: 'ERR_MODULE_NOT_FOUND'
}

Node.js v20.17.0
Enter fullscreen mode Exit fullscreen mode

If you bootstrap node using yarn it will work

base ❯ yarn node index.js
Human: Answer the following question to the best of your ability:
What is the answer to life, the universe, and everything?
Enter fullscreen mode Exit fullscreen mode

this is because here yarn will inject your .pnp.cjs loader into node, allowing node to resolve your packages via PnP.

Things I’ve learned along the way

  • Package managers matter. I never noticed the packageManager field in package.json or used CorePack to install a package manager. I didn’t know what CorePack was
  • Didn’t realize there such a thing as Yarn Classic (v1) using node_modules, and Yarn v2 / v3 / v4 using PnP
  • I didn’t know you need to use yarn to bootstrap node if you are working in PnP environment
  • Despite the fact that these things have been around for a while, stuff still breaks easily, and you always need to make sure you know what versions you are using, and see if upgrading them will fix your issues.

Conclusion

Latest versions of Yarn and PnP introduce a totally new way of resolving node modules. There is no node_modules folder anymore and it seems that the default version of Yarn that my system defaulted to (4.3.1) still has issues with how modules get resolved. Only after enabling the traceResolution I was able to see that it was looking for dependencies in node_modules , something it shouldn’t do when using PnP.

Top comments (0)