Scaling Codebases Without Platform Bloat

#softwareengineering #webdev #productivity #startup

Redundant compilation lags, bloated CI pipelines and version drift for internal
modules quietly drain developer velocity. And normally, it's too late when the
leadership gets the whiff of it. According to recent studies, the hidden tax on
your team looks exactly like this -

The Direct Time Drain: Your engineers waste an average of 25 to 50 minutes
due to compilation lags.
The Redundant Build: An engineer wastes his valuable time rebuilding
something that already exists and is being used in another team.
The Pipeline Blocker: Your Frontend Team spends an entire sprint cycle
completely idle, waiting for the Core Platform Team to version-tag and publish
the latest approved button changes.

All these, as bad as they sound, are common occurrences in the majority of
software engineering teams trying their best to ship fast and iterate faster.
The problems that I listed are not inevitable, and can be solved if we just
carefully design our engineering culture and the codebase that it revolves
around from the beginning.

In today's edition, let's tackle them one-by-one and continue where we left off
last time in our adventure of building
the ideal monorepo setup for
startups.

Assuming that you followed my advice of having a monorepo for all your codebase
needs for your startup, let's hit the road!

The Redundant Build: A guide to module re-use

While there's no sure-shot way of knowing what an engineer thinks before
embarking on the painstaking journey of rebuilding the wheel, we can mitigate
the risk through some primary cautions -

Build a collection of re-usable modules.
Business logic must never be allowed to be repeated.
Engineers should learn about the existing system for their use-case.

Though I know, forcing software engineers to do something they don't approve of
is never a good idea. But Companies and Team Leads should force these elementary
coding principles rigorously via code-reviews and documentation.

Building a Collection of Reusable modules

Startups that successfully keep growing, also tend to constantly rebuild many
parts of their system again and again. To give you a few examples -

UI Components: The same kind of button, the same navigation bar, the same
sidebar nav, etc. are rebuilt across different product lines to keep the look and
feel of the brand consistent. The website may be using the same color scheme as the
product dashboard, and hence the components get duplicated.
Backend Modules: If you have more than one backend, chances are that you
are duplicating the authentication, the logging, the middlewares, the utility
modules and many others depending upon your particular use-case.

Due to this, some parts of your website might look and feel different than the
other parts, worst-case it might feel they are built by different companies
altogether. And it's not just about how the UI looks, if you have duplicated
backend code, your team sometimes might end up debugging an edge-case over a
weekend that an engineer didn't think, while rebuilding the authentication
pipeline all over again.

To best avoid the above duplication, is to use the packages directory in your
monorepo root judiciously. That is if you followed the monorepo structure that
I advocated in the previous edition.

And if you are already doing that, it's better to keep the reusable modules kept
inside the packages directory - loosely coupled, highly parameterised and
customizable through environment variables and feature flags.

Consistent Business Logic

If there's one thing that you should absolutely avoid, then it has got to be
business logic duplication. Given enough time, codes change, bugs appear and get
fixed, engineers might come and go, but hardcore business logic seldom changes.

To give you an example, if you are building an accounting system, then your tax
calculation logic must never be repeated. If it does get repeated across your
backends and your frontends, then each update to the tax calculation logic,
debugging sessions and eventual hot-fixing, would need to be made across the
board and be forced to be compatible in each of the environments, causing
huge business consequences

I would like to recommend keeping a separate privileged module inside your
shared-modules, for such sensitive business logic and have a CODEOWNERS entry
such that unauthorised engineers (most likely the interns) can never make any
changes to the same.

An example CODEOWNERS entry might look something like this:

# .github/CODEOWNERS
# Guarding core business logic from unauthorized modifications

# Global fallback auditors
* @esence-io/platform-leads

# Strict guardrails for sensitive domain algorithms
/packages/domains/billing/tax-calculator.ts       @esence-io/finance-architects
/packages/domains/appointments/compliance.go     @esence-io/medical-directors

Documentation for Reusable Modules

This might be the only thing that separates an average organization from a
great organization that has some degree of foresight.

However, in an early-to-mid-stage startup, telling engineers to spend hours
writing long-form documentation on a separate Confluence or Notion workspace is
a fantasy. It creates out-of-date documentation platforms that nobody trusts,
adding to your organizational bloat.

The frugal, high-velocity alternative is Git-Adjacent Documentation:

Workspace READMEs: Every package inside your /packages/* directory
must contain a minimalist README.md explaining its API footprint, schema
rules, and runtime assumptions. If an engineer alters code logic, they update
the text in the exact same git commit string.
Strict Typing: Let your compiler act as your documentation
hub. By enforcing explicit input/output interface boundaries on shared
components, your IDE automatically tells the frontend engineer exactly what
parameters a component expects without them ever leaving their workspace
terminal window.
Better Code-Comments: Come-on, no one writes good comments.

If code isn't discoverable from inside the repo editor, it doesn't exist.
Keep it in git and the engineers will follow.

Trunk-Based Development: Unblocking the pipeline

What if every engineer works on the latest changes produced by every other team?
And what if every team starts taking ownership of the changes they make?

To answer both the questions in a strong Yes, your engineering team has to
follow the paradigm of "Trunk-Based Development".

Trunk-Based Development as opposed to module-versioning and importing foreign
modules from registries, means that a monorepo contains everything it needs and
every module it has, is on the latest version. And, cross-module dependencies
just become a question of either importing those modules or their DLLs or
binaries, completely bypassing the network-overhead of fetching latest modules.

There are no version numbers to increment, no private registries to maintain,
and no upstream breaking changes hidden behind a semver tag. And no wasteful
standup meetings discussing whether a change is breaking enough to bump up the
version entirely.

Once your company starts following trunk-based development:

If the Core Platform UI Team changes the button style or its parameters, it's
going to be the Core UI Team's responsibility to change it across the codebase.
Thereby, creating implicit ownership of all the changes that break integration
tests or that make other project's compilers scream.
The cost and complexity of infrastructure behind private module/package
registries instantly become zero and setting up the local environment for the
new joinee just becomes executing the build commands.
If a shared or an inter-team module breaks, no one has to get blocked for
getting fixes and updates.
All teams to some extent become software/module testers for all the other
teams by becoming a direct consumer with a zero-lag feedback loop.

One easy way to do it for TypeScript workspaces is by simply including workspace
dependencies like this:


  "dependencies": {
    "@packages/ui": "workspace:*"
  }

It isn't complicated, unless you start making it.

Compilation Lags and Caching Build Artifacts: With Nx

You pay money-tax to the government for earning, and pay time-tax to the
compiler for coding. Startups hire accountants for reducing tax paid to the
government, but seldom pay any attention to the tax paid to the compiler that
slowly leaks their pockets in terms of wasted productive hours of a developer
on a payroll.