DEV Community: David Arno

How I Built an Automated JS/TS Repository Analyzer in C#

David Arno — Fri, 24 Apr 2026 14:54:43 +0000

TL;DR

I built the JavaScript/TypeScript analysis engine for the Silverfish IDP — an Internal Developer Portal that automatically detects packaging tools, identifies component types, and extracts complete dependency graphs from repos. It handles monorepos, multiple lock file formats, and mixed JS/TS codebases—all without making assumptions about repo structure.

The Problem

At Silverfish Software, we're building an IDP that helps individual developers and engineering teams understand their entire codebase. But when you have hundreds of repositories spanning multiple languages, frameworks, and tools, how do you automatically make sense of it all?

For JavaScript and TypeScript repos specifically, the challenge is significant: every repo is different. Some use Yarn, others npm or pnpm. Some have monorepos with nested package.json files. Some mix JavaScript and TypeScript. Some have multiple lock files checked in (a real mess). And some don't have lock files at all.

I needed an analyzer that could handle all these cases automatically, with no manual configuration. No "please tell us which package manager you use" questions. Just point it at a repo and get back structured metadata about components, dependencies, and versions.

Step 1: Detect the Packaging Tool
The naive approach: Check if yarn.lock exists → use Yarn. Check if package-lock.json exists → use npm.

Reality is messier:

// Priority order matters
1. Check packageManager field in package.json ("yarn@4.1.0")
2. Look for lock files (yarn.lock, pnpm-lock.yaml, package-lock.json, bun.lock)
3. Check config files (.yarnrc.yml, pnpm-workspace.yaml)
4. Default to npm

The packageManager field was the key insight—it's set by corepack and is the source of truth. If it says Yarn, it's Yarn, even if npm somehow created a lock file too.

I also had to handle conflicts: I found real repos with both yarn.lock and package-lock.json checked in. My solution? Detect all of them, report the conflict, and parse only the highest-priority one.

public static async Task<PackagingToolDetectionResult> DetectAsync(
    IReadOnlyCollection<string> repoPaths,
    Func<string, Task<string?>> readFileContentAsync)
{
    // 1. Check packageManager field first
    var fromPackageManager = await TryDetectFromPackageManagerFieldAsync(...);
    if (fromPackageManager is not null) return fromPackageManager;

    // 2. Check lock files
    var fromLockFile = TryDetectFromLockFiles(...);
    if (fromLockFile is not null) return ...;

    // 3. Check config files
    var fromConfigFile = TryDetectFromConfigFiles(...);
    if (fromConfigFile is not null) return ...;

    // 4. Default to npm
    return new(PackagingTool.Npm, true);
}

Result: (PackagingTool.Yarn, LockFileNeedsGenerating: false) or similar.

Step 2: Identify Components and Their Type
Each package.json is a component. But what kind? And what does it do?

I classified each one into: Package (published to npm), Library (internal or private), and determined usage: Frontend, Backend, Fullstack, or Unknown.

The key was looking at dependencies:

static readonly HashSet<string> FrontendSignals = new() 
{ 
    "react", "vue", "@angular/core", "svelte", "react-router", "redux", ...
};

static readonly HashSet<string> BackendSignals = new()
{
    "express", "koa", "mongoose", "pg", "apollo-server", "prisma", ...
};

// If a package depends on react + express = fullstack
// If only react = frontend
// If only express = backend

I also extracted language info:

// Pure JS? Check for no TypeScript signals
// TypeScript? Look for typescript pkg + @types/*
// Mixed? Has flow-bin + typescript OR tsconfig.json's allowJs = true

And pulled in version constraints:

// Node version: from engines.node in package.json or .nvmrc file
// TS version: from devDependencies
// ECMAScript target: from tsconfig.json compilerOptions

Result: A JsComponent record with all metadata attached—used by Silverfish's dashboard to display component details instantly.

Step 3: Parse Lock Files (The Hard Part)
This was the gnarly part. Four different formats, each with quirks.

Yarn Lock (v1 Classic)
Looks like TOML with nested dependency lists:

"@pkgjs/parseargs@^0.11.0":
  version "0.11.0"
  resolved "https://registry.npmjs.org/..."
  dependencies:
    package-json "^6.0.0"

I wrote a line-by-line parser. The trick: track indentation to know when you're inside a package block vs. dependency list.

npm package-lock.json
Flat JSON structure (v2/v3):

{
  "packages": {
    "node_modules/lodash": {
      "version": "4.17.21",
      "dependencies": { ... }
    }
  }
}

Easier to parse with JsonDocument, but the key names have node_modules/ prefixes that need stripping.

pnpm-lock.yaml
YAML with name@version keys:

packages:
  /lodash/4.17.21:
    version: 4.17.21
    dependencies:
      react: 18.2.0

I treated this as mostly line-based text parsing since I didn't want to add a full YAML dependency. Works for the common cases.

Bun Lock
JSONC format with array-based entries. Least common, so I parse it but mark binary bun.lockb files as unparseable.

Step 4: Resolve Dependencies
Once I had a parsed lock file, I needed to extract:

Local dependencies (internal workspace packages like @company/shared)
Direct dependencies (what's explicitly in package.json)
Transitive dependencies (what your dependencies need)

// Read package.json dependencies
var directRanges = ReadDirectDependencyRanges(packageJsonContent);

// For each direct dep, look it up in the lock file
foreach (var (name, range) in directRanges)
{
    var pkg = Resolve(name, range, parsedLock);
    if (pkg != null)
    {
        // It's resolved to version X.Y.Z
        direct.Add(new ResolvedDependency(pkg.Name, pkg.Version, range));

        // Queue it to traverse its dependencies
        queue.Enqueue(pkg);
    }
}

// Depth-first traversal to collect transitives
while (queue.TryDequeue(out var pkg))
{
    foreach (var (depName, depRange) in pkg.DependencyRanges)
    {
        var dep = Resolve(depName, depRange, parsedLock);
        if (dep != null && !visited.Contains($"{dep.Name}@{dep.Version}"))
        {
            transitive.Add(...);
            queue.Enqueue(dep);
        }
    }
}

Result: Three lists of ResolvedDependency objects with exact versions and requested ranges. The Silverfish dashboard uses this to build the full dependency graph in its UI.

Step 5: Handle Monorepos
Monorepos have multiple package.json files. The key insight: walk up the directory tree to find the root lock file.

static IEnumerable<string> AncestorDirs(string dir)
{
    var current = dir;
    while (true)
    {
        yield return current;
        if (string.IsNullOrEmpty(current)) break;
        current = Path.GetDirectoryName(current);
    }
}

So packages/web/package.json in an entria-style monorepo correctly finds the root yarn.lock instead of failing. Each workspace member gets its own component record in Silverfish.

How Silverfish Uses This
Once the analyzer extracts all this metadata, Silverfish:

Maps dependencies visually — showing which components depend on what Flags version mismatches — when different packages pin different versions of the same library
Detects tech stacks — knowing which services are frontend, which are backend, which databases they use
Tracks upgrades — identifying outdated packages and planning coordinated updates
Enables governance — enforcing policies like "no direct jquery dependencies" or "all frontends must use React 18+"

Lessons Learned

Format-specific parsing is worth it: I could have given up on Yarn/pnpm/Bun and only parsed npm lock files. But each format's parser is ~100-150 lines and handles real repos that exist in the wild.
Conflicts are data, not errors: Instead of failing when I find multiple lock files, I report them. That's valuable information ("why do you have both yarn.lock and package-lock.json?").
Monorepos are normal: Walking ancestor directories for lock files + detecting internal workspace packages turned out to be essential, not an edge case.
Version constraints matter: Storing both the requested range (^1.2.3) and resolved version (1.2.5) proved useful—you can detect upgradeable deps without breaking changes.

What's Next
The JS/TS analyzer is one piece of Silverfish's language support. I'll building similar analyzers for Python, Go, Java, and other ecosystems. The pattern is the same: detect the package manager, identify components, resolve dependencies, extract versions.

If you're trying to understand complex multi-language codebases at scale, this approach should help. The code is C# 14 with only standard library dependencies—no bloat.

Check out the Silverfish Dashboard to see the analyzer in action, or hit me up if you have questions about the implementation.

Silverfish IDP: A low cost, language‑first Internal Developer Portal for understanding your software estate

David Arno — Thu, 16 Apr 2026 15:29:54 +0000

Today I’m releasing the first public version of the Silverfish IDP, an Internal Developer Portal designed to help developers and engineering teams understand the structure of their software estates — not just at the repository level, but at the component and dependency level too.

The Silverfish IDP takes a language‑first approach. It starts with repository discovery and grows into a broader platform for engineering insight and governance. The aim is to give teams clarity without without needing a complex platform rollout and without signing up to $1000s a year for the privilege.

The Silverfish IDP also takes an affordability‑first approach. It is completely free to use - and will remain so - for anyone just wanting to analyse public repos. I've built the product using a range of open source tools and libraries and this is just a way of me "paying forward" for the efforts of others.

What’s included in the first release

GitHub‑based sign‑in and onboarding
Workspaces and Organizations
Repository mapping + scan‑state control
Hierarchy naming schemes
Component and dependency discovery for supported languages (the .NET languages, C#, F# & VB.Net, and Ruby)
In‑product documentation

Why I built it

Teams often know what they’ve built, but not always how it fits together. And sometimes, they aren't really sure what they have even built. Things are created, the developer who created it moves on and well, let's be honest with each other, devs aren't always the best at documenting what they did. Silverfish aim to address both of these issues by making software structure visible and navigable and by providing information on the contents of each repo to help identify what is is and what it does.

What’s next

Short‑term improvements include more languages, better visualisations, hierarchy support, and workspace/org membership.

Medium‑term plans include quality rules, reporting, release tracking, and support for Azure DevOps, Bitbucket, and Codeberg.

Longer‑term: private repo support, GitLab, Stackhero, and richer metrics including selected DORA‑style measures.

It’s early days, but the foundation is solid — and I’d love feedback from the Dev.to community as it evolves.