What is a Polyglot Monorepo?
A polyglot monorepo is a single Git repository containing services and packages written in multiple programming languages. The alternative — a polyrepo — means one repo per service, per team, per language. When your frontend team, your Go API team, and your Python ML team all operate in separate repos, sharing contracts between them becomes a coordination problem.
This repo is beacon-monorepo. It's a real-time analytics platform. It contains:
beacon-monorepo/
├── frontend/
│ └── web/ ← Next.js dashboard (TypeScript)
├── services/
│ ├── api/ ← REST API (Go)
│ ├── ingest/ ← Data ingestion service (Go)
│ ├── ml/ ← ML inference API (Python / FastAPI)
│ └── worker/ ← Background jobs (Python / Celery)
├── packages/
│ ├── ui/ ← Shared React components (TypeScript)
│ └── sdk/ ← TypeScript client SDK
├── pkg/
│ ├── shared/ ← Go shared utilities
│ └── store/ ← Go data access layer
├── libs/
│ └── shared/ ← Python shared models + Pydantic schemas
├── proto/ ← Protobuf definitions (source of truth)
├── gen/
│ ├── go/ ← Generated Go stubs
│ ├── python/ ← Generated Python stubs
│ └── ts/ ← Generated TypeScript stubs
├── Taskfile.yml ← Root task orchestration (cross-language)
├── pnpm-workspace.yaml
├── package.json ← Root (biome, lefthook, turbo)
├── pyproject.toml ← uv workspace root
├── uv.lock
├── biome.json
├── .golangci.yml
└── go.work ← (gitignored — local dev only)
The core idea: services are owned by different teams writing different languages, but they share contracts defined in proto/, config defined at the repo root, and a single task runner (Taskfile.yml) that orchestrates everything with one set of commands.
Tool 1: Three Workspace Managers — The Parallel Foundation
There is no single package manager that handles TypeScript, Python, and Go. Instead, three workspace managers coexist in the same repo — each governing its own language, each completely ignorant of the others.
| Language | Manager | Config file | Dep linking mechanism |
|---|---|---|---|
| TypeScript | pnpm workspaces | pnpm-workspace.yaml |
workspace:* symlinks |
| Python | uv workspaces | root pyproject.toml
|
{ workspace = true } editable installs |
| Go | go workspaces | go.work |
local module overlay |
They coexist peacefully because they key on different file extensions. pnpm reads package.json. uv reads pyproject.toml. Go reads go.mod. A directory like services/ml/ that contains a pyproject.toml is invisible to pnpm. A directory with only a package.json is invisible to uv.
TypeScript: pnpm-workspace.yaml
packages:
- 'frontend/*'
- 'packages/*'
- 'gen/ts' # generated TypeScript stubs — must be a workspace member
gen/ts/ must be a workspace member, not a file: dependency. A file: reference copies the package into node_modules at install time — after buf generate regenerates the stubs, the copies are stale until you re-run pnpm install. A workspace:* reference is a symlink — always live.
packages/sdk/package.json depends on the shared UI and the generated TypeScript stubs:
{
"name": "@beacon/sdk",
"dependencies": {
"@beacon/ui": "workspace:*",
"@beacon/proto-ts": "workspace:*" // symlinked — reflects buf generate immediately
}
}
Python: root pyproject.toml
[project]
name = "beacon-root"
version = "0.1.0"
requires-python = ">=3.12"
[tool.uv.workspace]
# List Python members explicitly — globs must not match dirs without pyproject.toml
members = [
"services/ml",
"services/worker",
"libs/shared",
"gen/python", # generated proto stubs — must be listed so uv can resolve them
]
[tool.uv.sources]
shared = { workspace = true } # the workspace:* equivalent for Python
services/ml/pyproject.toml declares its internal dep:
[project]
name = "ml"
dependencies = [
"shared", # internal — resolves via workspace
"fastapi>=0.115",
"torch>=2.3",
]
[tool.uv.sources]
shared = { workspace = true }
Why explicit members, not globs? uv errors if a glob matches a directory without a pyproject.toml. frontend/web/ and packages/ui/ have package.json but no pyproject.toml — a glob like packages/* would break uv. Be explicit.
Go: go.work
go 1.23
use (
./services/api
./services/ingest
./pkg/shared
./pkg/store
./gen/go ← generated proto stubs (own go.mod)
)
Critical: add go.work and go.work.sum to .gitignore. CI must validate each Go module against its own go.mod independently. The workspace is a local dev convenience — not a CI mechanism.
go.mod plumbing — how CI works without go.work
This is the part most polyglot monorepo guides skip. When CI runs GOWORK=off, each Go module must declare its in-repo dependencies in its own go.mod using replace directives — otherwise the module can't find gen/go or pkg/shared.
services/api/go.mod:
module github.com/your-org/beacon/services/api
go 1.23
require (
github.com/your-org/beacon/gen/go v0.0.0
github.com/your-org/beacon/pkg/shared v0.0.0
// external deps...
)
// replace directives make GOWORK=off CI work:
// each module explicitly maps its in-repo deps to disk paths
replace (
github.com/your-org/beacon/gen/go => ../../gen/go
github.com/your-org/beacon/pkg/shared => ../../pkg/shared
)
go work sync propagates the require versions from each go.mod. The replace directives work with or without go.work — so GOWORK=off CI and go.work local dev both resolve to the same local disk paths.
Run go mod tidy in each module after adding the replace directives. The v0.0.0 version is a placeholder that go mod tidy fills in as a pseudo-version.
# .gitignore
go.work
go.work.sum
.task/
.venv/
node_modules/
.turbo/
.next/
__pycache__/
*.pyc
services/*/bin/
What uv sync and pnpm install and go work sync each do
pnpm install # links TypeScript packages via symlinks in node_modules
uv sync --group dev # installs all Python workspace members as editable installs
go work init # creates go.work on a fresh clone (only needed once — it's gitignored)
go work use ./services/api ./services/ingest ./pkg/shared ./pkg/store ./gen/go
go work sync # syncs go.mod files across Go modules (run after adding new deps)
Fresh clone problem: go.work is gitignored, so a fresh clone has no workspace file. The go work init + go work use steps only run once per machine. The Go Taskfile handles this automatically.
The key insight: these three commands are completely independent. Running pnpm install does not affect Python deps. Running uv sync does not touch node_modules. None of them touch each other. The three workspace managers are parallel foundations — they don't compose, they coexist.
Tool 2: Task (Taskfile.yml) — The Cross-Language Orchestrator
Three workspace managers handle dependencies. Task handles tasks (build, test, lint, generate, dev). It's the single entry point for everything in the repo, regardless of language.
Task is a language-agnostic binary. Install it once:
brew install go-task
# or: sh -c "$(curl --location https://taskfile.dev/install.sh)" -- -d -b ~/.local/bin
The root Taskfile.yml
version: '3'
includes:
ts:
taskfile: ./Taskfile.ts.yml
optional: true
py:
taskfile: ./Taskfile.py.yml
optional: true
go:
taskfile: ./Taskfile.go.yml
optional: true
proto:
taskfile: ./Taskfile.proto.yml
optional: true
tasks:
doctor:
desc: Verify all required tools are installed
cmds:
- command -v node || (echo "node not found — https://nodejs.org" && exit 1)
- command -v pnpm || (echo "pnpm not found — npm install -g pnpm" && exit 1)
- command -v uv || (echo "uv not found — https://docs.astral.sh/uv" && exit 1)
- command -v go || (echo "go not found — https://go.dev/dl" && exit 1)
- command -v buf || (echo "buf not found — https://buf.build/docs/cli" && exit 1)
- command -v jq || (echo "jq not found — brew install jq" && exit 1)
- echo "All tools present"
setup:
desc: Install all dependencies (run task doctor first)
deps: [doctor]
cmds:
- pnpm install
- uv sync --group dev # single .venv at repo root; --group dev includes pytest/mypy/ruff
- task: go:init # creates go.work if missing (idempotent)
- go work sync
- task: proto:generate
build:all:
desc: Build everything in dependency order
cmds:
- task: proto:generate # proto first — all languages depend on it
- task: ts:build
- task: go:build
- task: py:build
test:all:
desc: Test all languages in parallel
deps:
- ts:test
- go:test
- py:test
lint:all:
desc: Lint all languages in parallel
deps:
- ts:lint
- go:lint
- py:lint
dev:
desc: Start all dev servers
deps:
- ts:dev
- go:dev
- py:dev
Language-specific Taskfiles
Taskfile.ts.yml — TypeScript:
version: '3'
tasks:
build:
sources:
- 'frontend/**/*.ts'
- 'frontend/**/*.tsx'
- 'packages/**/*.ts'
- '**/package.json' # dep changes should bust the cache
- 'pnpm-lock.yaml'
generates: ['frontend/web/.next/**']
cmds:
- pnpm turbo run build
lint:
sources: ['frontend/**/*.ts', 'packages/**/*.ts', '**/package.json']
generates: ['.task/ts-lint.stamp']
cmds:
- pnpm biome check .
- touch .task/ts-lint.stamp
test:
sources:
- 'frontend/**/*.ts'
- 'packages/**/*.ts'
- 'frontend/**/*.test.ts'
- 'pnpm-lock.yaml'
cmds:
- pnpm turbo run test
dev:
cmds:
- pnpm turbo run dev --filter=@beacon/web
Taskfile.go.yml — Go:
version: '3'
# Prerequisite: jq must be installed (brew install jq / apt install jq)
tasks:
init:
desc: Create go.work for local development (gitignored; run once per clone)
status:
- test -f go.work # skip if go.work already exists
cmds:
- go work init
- go work use ./services/api ./services/ingest ./pkg/shared ./pkg/store ./gen/go
build:
deps: [init] # ensures go.work exists before building
sources:
- 'services/api/**/*.go'
- 'services/ingest/**/*.go'
- 'pkg/**/*.go'
- '**/go.mod' # dep changes should bust the cache
- '**/go.sum'
generates: ['services/api/bin/api', 'services/ingest/bin/ingest']
cmds:
- cd services/api && go build -o bin/api ./cmd/api
- cd services/ingest && go build -o bin/ingest ./cmd/ingest
lint:
deps: [init]
sources: ['services/**/*.go', 'pkg/**/*.go']
generates: ['.task/go-lint.stamp']
cmds:
- go work edit -json | jq -r '.Use[].DiskPath' |
xargs -P4 -I{} sh -c 'cd {} && golangci-lint run ./...'
- touch .task/go-lint.stamp
test:
deps: [init]
sources: ['services/**/*.go', 'pkg/**/*.go']
cmds:
- go work edit -json | jq -r '.Use[].DiskPath' |
xargs -P4 -I{} sh -c 'cd {} && go test ./... -race'
dev:
deps: [init]
cmds:
- cd services/api && go run ./cmd/api
Taskfile.py.yml — Python:
version: '3'
tasks:
build:
sources:
- 'services/ml/**/*.py'
- 'services/worker/**/*.py'
- 'libs/**/*.py'
- '**/pyproject.toml' # dep changes should bust the cache
- 'uv.lock'
generates: ['.task/py-build.stamp']
cmds:
- uv sync --group dev
- touch .task/py-build.stamp
lint:
sources: ['services/ml/**/*.py', 'services/worker/**/*.py', 'libs/**/*.py', 'uv.lock']
generates: ['.task/py-lint.stamp']
cmds:
- uv run ruff check services/ml services/worker libs
- uv run ruff format --check services/ml services/worker libs
- uv run mypy services/ml/ services/worker/ libs/ # from repo root — not per-service
- touch .task/py-lint.stamp
test:
sources: ['services/ml/**/*.py', 'services/worker/**/*.py', 'libs/**/*.py']
cmds:
- uv run pytest services/ml services/worker libs -x
dev:
cmds:
- uv run fastapi dev services/ml/src/ml/main.py
Key concepts:
-
deps:runs tasks in parallel —test:allruns all three test suites simultaneously -
cmds:runs sequentially —build:allruns proto first, then the rest -
sources/generatesis Task's caching: if source files haven't changed since the last run (hash stored in.task/), the task is skipped entirely - Add
.task/to.gitignore -
task build:allrequires all three toolchains installed (node + pnpm, uv, go, buf, jq). Frontend-only or backend-only developers who run it without the full stack will get a "task not found" error. Document this in your repo README — or splittask build:ts,task build:go,task build:pyas the normal developer interface.
CI: affected-only runs
Task has no built-in --affected. In CI, detect changed language areas with git diff:
# ci/affected.sh
changed=$(git diff --name-only origin/main...HEAD)
echo "$changed" | grep -qE '^(frontend|packages)/' && task ts:test
echo "$changed" | grep -qE '^(services/(api|ingest)|pkg)/' && task go:test
echo "$changed" | grep -qE '^(services/(ml|worker)|libs)/' && task py:test
echo "$changed" | grep -qE '^proto/' && {
task proto:generate
task ts:test && task go:test && task py:test # proto change affects all
}
A PR touching only services/ml/ won't re-run Go tests.
Tool 3: buf + Protobuf — The Contract Layer
This is the tool that makes a polyglot monorepo genuinely worth the complexity. A single .proto file becomes the source of truth for how your TypeScript frontend, your Go API, and your Python ML service all communicate. Change the proto once — regenerate — all three languages are in sync.
buf.yaml — workspace config at repo root
# buf.yaml
version: v2
modules:
- path: proto/analytics
name: buf.build/your-org/analytics
- path: proto/events
name: buf.build/your-org/events
deps:
- buf.build/googleapis/googleapis
lint:
use: [STANDARD]
except: [PACKAGE_DIRECTORY_MATCH]
breaking:
use: [FILE]
buf.gen.yaml — one command, three languages
# buf.gen.yaml
version: v2
inputs:
- directory: proto/analytics
- directory: proto/events
plugins:
# Go: message stubs
- remote: buf.build/protocolbuffers/go
out: gen/go
opt: paths=source_relative
# Go: gRPC service stubs
- remote: buf.build/grpc/go
out: gen/go
opt: paths=source_relative
# Python: message stubs
- remote: buf.build/protocolbuffers/python
out: gen/python
# Python: gRPC service stubs
- remote: buf.build/grpc/python
out: gen/python
# Python: mypy type stubs (community plugin)
- remote: buf.build/community/nipunn1313-mypy
out: gen/python
# TypeScript: Protobuf-ES (modern, works with Connect-RPC)
- remote: buf.build/bufbuild/es
out: gen/ts
opt: target=ts
managed:
enabled: true
override:
- file_option: go_package_prefix
value: github.com/your-org/beacon/gen/go
Run buf generate to regenerate all three at once.
Generated code as first-class modules
Each generated directory is its own module with its own manifest — so the three workspace managers can find it:
gen/
├── go/
│ └── go.mod ← module github.com/your-org/beacon/gen/go
│ each consumer's go.mod has: replace ... => ../../gen/go
├── python/
│ └── pyproject.toml ← listed in uv workspace members; consumers use { workspace = true }
└── ts/
└── package.json ← listed in pnpm-workspace.yaml; consumers use workspace:*
All three use the same pattern as other internal packages — workspace linking, not file: copies. gen/go/ is also listed in go.work for local dev, and each consumer's go.mod has a replace directive for CI.
# Taskfile.proto.yml
tasks:
generate:
sources:
- proto/**/*.proto
generates:
- gen/go/**/*.go
- gen/python/**/*_pb2.py
- gen/ts/**/*_pb.ts
cmds:
- buf lint
- buf generate
- buf breaking --against '.git#branch=main'
Why this matters: the breaking change check (buf breaking) is the safety net. If a Go developer renames a proto field, buf breaking fails the build before the Python and TypeScript consumers break. The contract is enforced at the definition layer, not the consumer layer.
Tool 4: Per-Language Root Config — Three Files, One Repo
Each language has one config file at the repo root. No duplication across services, no "why is lint disabled in this one service" surprises.
biome.json — TypeScript
{
"$schema": "https://biomejs.dev/schemas/1.9.0/schema.json",
"linter": {
"rules": {
"correctness": {
"noUnusedVariables": "error",
"noUnusedImports": "error"
},
"style": { "useConst": "error" }
}
},
"formatter": {
"indentStyle": "space",
"indentWidth": 2,
"lineWidth": 100
},
"files": {
"ignore": ["gen/ts/**", "**/node_modules/**", "**/.next/**"]
}
}
root pyproject.toml — Python lint + type config
[tool.ruff]
line-length = 88
target-version = "py312"
src = ["src"]
exclude = ["gen/python"]
[tool.ruff.lint]
select = ["E", "W", "F", "I", "B", "UP"]
ignore = ["E501"]
fixable = ["ALL"]
[tool.ruff.lint.isort]
known-first-party = ["shared", "ml", "worker"]
[tool.mypy]
python_version = "3.12"
strict = true
mypy_path = "$MYPY_CONFIG_FILE_DIR/libs/shared/src"
exclude = ["gen/python"]
[[tool.mypy.overrides]]
module = ["*_pb2", "*_pb2_grpc"]
ignore_errors = true # suppress errors in generated proto stubs
[dependency-groups]
dev = [
"pytest>=8",
"pytest-asyncio>=0.25",
"mypy>=1.15",
"ruff>=0.9",
]
Critical — mypy invocation: mypy reads config from the directory it's invoked from, not from the file being checked. Always run uv run mypy services/ml/ services/worker/ libs/ from the repo root. Running from inside services/ml/ silently ignores strict = true and falls back to defaults.
.golangci.yml — Go
version: "2"
run:
timeout: 5m
tests: true
linters:
default: none
enable:
- errcheck
- staticcheck
- unused
- ineffassign
- govet
- revive
- gocritic
- misspell
- gosec
- bodyclose
- copyloopvar
exclusions:
generated: strict
rules:
- path: "gen/go/.*\\.pb\\.go$"
linters: [revive, gocritic, godot, errcheck]
- path: "_test\\.go$"
linters: [gosec, errcheck]
formatters:
enable:
- gofumpt
- goimports
settings:
goimports:
local-prefixes:
- github.com/your-org/beacon
.editorconfig — baseline cross-language consistency
Biome, Ruff, and golangci-lint enforce formatting within each language, but editors need .editorconfig for baseline consistency before any tool runs — charset, line endings, trailing newlines:
# .editorconfig
root = true
[*]
charset = utf-8
end_of_line = lf
insert_final_newline = true
trim_trailing_whitespace = true
[*.go]
indent_style = tab
indent_size = 4
[*.{ts,tsx,js,json,yaml,yml,toml}]
indent_style = space
indent_size = 2
[*.py]
indent_style = space
indent_size = 4
[*.proto]
indent_style = space
indent_size = 2
Why this matters: three config files, all at the repo root. Each tool finds its own by walking up the directory tree. You can change the Go linting rules for every Go service in one edit, the Python formatting rules for every Python service in one edit, and the TypeScript rules for every TypeScript package in one edit. No per-service drift.
Tool 5: Three Linters, One task lint Command
Each language uses its own best-in-class linter. Task unifies them:
task lint:all # runs all three in parallel (deps: is parallel)
task ts:lint # TypeScript only
task go:lint # Go only
task py:lint # Python only
Why not one linter for everything?
There is no cross-language linter that handles TypeScript + Python + Go well. The best tools are language-specific:
- Biome — TypeScript: sub-millisecond, replaces ESLint + Prettier
- Ruff — Python: 10–100x faster than Flake8, replaces isort + Black
- golangci-lint — Go: aggregates 50+ linters behind one binary
Task makes them feel like one: task lint:all exits non-zero if any of them fails, regardless of language.
Build graph — how proto flows into all three
proto/analytics/*.proto
proto/events/*.proto
↓ buf generate
↓
gen/go/ gen/python/ gen/ts/
↓ ↓ ↓
pkg/shared libs/shared packages/sdk
↓ ↓ ↓
services/api services/ml frontend/web
services/ingest services/worker
A change to proto/ forces all three build chains to run. Everything else is independent — a change to services/api/ doesn't rerun Python tests.
CI: three parallel language jobs
# .github/workflows/ci.yml
jobs:
detect:
runs-on: ubuntu-latest
outputs:
proto: ${{ steps.changes.outputs.proto }}
ts: ${{ steps.changes.outputs.ts }}
go: ${{ steps.changes.outputs.go }}
py: ${{ steps.changes.outputs.py }}
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: dorny/paths-filter@v3
id: changes
with:
filters: |
proto:
- 'proto/**'
ts:
- 'frontend/**'
- 'packages/**'
- 'gen/ts/**'
- 'proto/**' # proto change triggers TS too
go:
- 'services/api/**'
- 'services/ingest/**'
- 'pkg/**'
- 'gen/go/**'
- 'proto/**'
py:
- 'services/ml/**'
- 'services/worker/**'
- 'libs/**'
- 'gen/python/**'
- 'proto/**'
proto:
needs: detect
if: needs.detect.outputs.proto == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: |
npm install -g @bufbuild/buf
buf lint && buf generate
buf breaking --against '.git#branch=main'
ts:
needs: detect
if: needs.detect.outputs.ts == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0 # required for turbo --affected
- uses: pnpm/action-setup@v4
- run: pnpm install --frozen-lockfile # equivalent of uv sync --frozen
- run: pnpm turbo run typecheck lint test --affected
go:
needs: detect
if: needs.detect.outputs.go == 'true'
runs-on: ubuntu-latest
strategy:
matrix:
module: [services/api, services/ingest, pkg/shared, pkg/store]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-go@v5
with:
go-version-file: ${{ matrix.module }}/go.mod
cache-dependency-path: ${{ matrix.module }}/go.sum
# GOWORK=off is explicit — not relying on go.work being gitignored
# This forces each module to prove it has complete go.mod declarations
- run: cd ${{ matrix.module }} && GOWORK=off golangci-lint run ./...
- run: cd ${{ matrix.module }} && GOWORK=off go test ./... -race
py:
needs: detect
if: needs.detect.outputs.py == 'true'
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v5
with:
enable-cache: true
- run: uv sync --frozen --group dev
# --frozen: fail if uv.lock is out of date (never silently update in CI)
# --group dev: includes pytest, mypy, ruff
- run: uv run ruff check services/ml/ services/worker/ libs/
- run: uv run mypy services/ml/ services/worker/ libs/
# always from repo root — mypy does not discover config per-file like Ruff
- run: uv run pytest services/ml/ services/worker/ libs/ -x
- run: uv cache prune --ci
A PR touching only services/ml/ skips the proto, ts, and go jobs entirely.
Tool 6: lefthook — One Hook to Rule All Three Languages
Each language article recommended its own hook tool (custom .githooks/ for TypeScript, pre-commit for Python, lefthook for Go). In a polyglot repo, you pick one. lefthook wins: it's a single binary, works with all languages, and runs hooks in parallel by default.
Install
lefthook is managed as a root-level dev dependency so everyone on the team gets it:
// package.json (root)
{
"devDependencies": {
"@biomejs/biome": "^1.9.0",
"lefthook": "^1.11.0",
"turbo": "^2.0.0"
},
"scripts": {
"prepare": "lefthook install"
}
}
pnpm install runs prepare automatically — lefthook hooks are wired up on first install.
lefthook.yml
# lefthook.yml
glob_matcher: doublestar # ** matches 0 or more dirs (required for nested files)
pre-commit:
parallel: true
jobs:
- name: ts-lint
glob: "**/*.{ts,tsx}"
run: pnpm biome check --no-errors-on-unmatched {staged_files}
- name: py-lint
glob: "**/*.py"
run: uv run ruff check {staged_files}
stage_fixed: true
- name: py-format
glob: "**/*.py"
run: uv run ruff format {staged_files}
stage_fixed: true
- name: proto-lint
glob: "**/*.proto"
run: buf lint
pre-push:
parallel: false
jobs:
- name: go-lint
glob: "**/*.go"
# golangci-lint cannot lint files across multiple packages (named files must all
# be in one directory — see golangci-lint#3715). Run per-module instead:
run: go work edit -json | jq -r '.Use[].DiskPath' |
xargs -P4 -I{} sh -c 'cd {} && GOWORK=off golangci-lint run ./...'
- name: go-mod-tidy
glob: "go.{mod,sum}"
# go mod tidy is per-module in a multi-module workspace; iterate over all modules:
run: go work edit -json | jq -r '.Use[].DiskPath' |
xargs -I{} sh -c 'cd {} && GOWORK=off go mod tidy && git diff --exit-code go.mod go.sum'
- name: proto-breaking
glob: "**/*.proto"
# Guard against first push / no origin/main (new repos, shallow clones)
run: |
if git rev-parse origin/main >/dev/null 2>&1; then
buf breaking --against '.git#branch=main'
else
echo "No origin/main — skipping breaking change check"
fi
- name: py-typecheck
glob: "**/*.py"
# mypy is 10-30s on real codebases — too slow for pre-commit, right for pre-push
# pass whole dirs, not {staged_files}: mypy needs package-level context
run: uv run mypy services/ml/ services/worker/ libs/ --config-file pyproject.toml
Key concepts:
-
glob_matcher: doublestar— without this,**/*.pywon't matchservices/ml/src/ml/main.py(two directories deep). This single line makes nested file matching work correctly. - Each job is skipped entirely if no staged files match its glob — committing only TypeScript changes doesn't run
go-lintat all. -
stage_fixed: true— after Ruff auto-fixes a Python file, lefthook re-stages it so the fix is included in the commit. -
parallel: true— all four language checkers run simultaneously. On a fast machine, a commit touching files in all three languages still completes in the time of the slowest single checker. -
pre-pushrunsbuf breaking— the breaking change check is too slow for pre-commit but important before pushing to a shared branch.
The key insight: you don't need different hook tools for different languages. lefthook routes by file glob. The Python and Go developers on your team never need to know that Biome exists; the TypeScript developers never need to know that Ruff exists. They all run pnpm install once and get the right checks for their files automatically.
The Full Picture: How These Tools Interact
Developer commits
↓
lefthook pre-commit (all four checks in parallel)
↓ Biome on staged .ts/.tsx files
↓ Ruff on staged .py files (+ auto-fix + re-stage)
↓ golangci-lint on staged .go files
↓ buf lint on staged .proto files
↓ (any failure → abort commit; all pass → continue)
↓
Three workspace managers resolve internal deps in parallel:
↓ pnpm: workspace:* symlinks for TypeScript
↓ uv: { workspace = true } editable installs for Python
↓ go work: local module overlay for Go (gitignored)
↓
task build:all
↓ proto:generate (buf → gen/go/ + gen/python/ + gen/ts/)
↓ ts:build (pnpm turbo — dep-ordered, cached in .turbo/)
↓ go:build (per-module, cached in .task/)
↓ py:build (uv sync, cached in .task/)
↓
CI: detect-changes → three parallel language jobs
↓ proto change → all three jobs + breaking check
↓ TypeScript: pnpm turbo --affected (fetch-depth: 0)
↓ Go: GOWORK=off → matrix per module → go test -race
↓ Python: uv run pytest (affected services)
↓ all jobs required → single ci-complete gate
Summary: Why This Stack
| Problem | Tool | Mechanism |
|---|---|---|
| TypeScript dep linking | pnpm workspaces |
workspace:* + symlinks |
| Python dep linking | uv workspaces |
{ workspace = true } + editable installs |
| Go dep linking | go workspaces |
use directives (gitignored) |
| Cross-language task orchestration | Task (Taskfile.yml) |
includes + sources/generates caching |
| Cross-language contracts | buf + Protobuf | single .proto → 3 language stubs |
| TypeScript lint + format | Biome | root biome.json
|
| Python lint + format | Ruff | root pyproject.toml [tool.ruff]
|
| Go lint + format | golangci-lint + gofumpt | root .golangci.yml
|
| Cross-language git hooks | lefthook |
glob_matcher: doublestar + parallel jobs |
| CI: only run changed languages | dorny/paths-filter | per-language path filters + job conditions |
The polyglot monorepo isn't one tool — it's a composition. Three workspace managers run in parallel. Task sits on top as the single command interface. buf provides the shared contract layer between all three languages. lefthook enforces quality at commit time regardless of which language a developer touches.
The complexity cost is real: you're maintaining three build toolchains instead of one. The payoff is equally real: one git blame, one pull request for a cross-language feature, one place to define the contract between your TypeScript frontend and your Go and Python backends.
Deploying a Python Service from the Polyglot Monorepo
Go compiles to a static binary — Docker is trivial. TypeScript builds to a dist/ directory — straightforward. Python is the tricky one: editable workspace installs point back at source paths and break inside a container without extra work.
The fix is uv sync --package ml --no-editable. The build context must be the repo root — not services/ml/ — because uv sync needs to resolve libs/shared, gen/python, and the root pyproject.toml.
Without a .dockerignore, Docker sends the entire monorepo as context including node_modules/, Go binaries, and the frontend build cache. Add this at the repo root:
# .dockerignore (at repo root — used by all Python service Dockerfiles)
frontend/
packages/
node_modules/
.turbo/
.task/
.next/
*.go
go.*
go.work
go.work.sum
.venv/
Then build from the repo root: docker build -f services/ml/Dockerfile .
# services/ml/Dockerfile
FROM python:3.12-slim AS builder
COPY --from=ghcr.io/astral-sh/uv:latest /uv /usr/local/bin/uv
WORKDIR /app
# Copy manifests first — dep install layer is cached separately from source
COPY pyproject.toml uv.lock ./
COPY libs/shared/pyproject.toml libs/shared/pyproject.toml
COPY gen/python/pyproject.toml gen/python/pyproject.toml
COPY services/ml/pyproject.toml services/ml/pyproject.toml
RUN uv sync \
--package ml \ # install only ml + its deps (libs/shared, gen/python, etc.)
--no-dev \ # exclude pytest, mypy, ruff
--frozen \ # fail if uv.lock is stale
--no-editable # copy source into site-packages — no source tree needed at runtime
# Copy source after dep install (changes here don't bust dep cache)
COPY libs/shared/src libs/shared/src
COPY gen/python gen/python
COPY services/ml/src services/ml/src
FROM python:3.12-slim
COPY --from=builder /app/.venv /app/.venv
ENV PATH="/app/.venv/bin:$PATH"
CMD ["uvicorn", "ml.main:app", "--host", "0.0.0.0", "--port", "8080"]
--no-editable is the critical flag. Without it, libs/shared installs as an editable install pointing at libs/shared/src/ in the build context. The final image has a .venv with broken symlinks and imports fail at runtime. --no-editable copies the package source into .venv/lib/pythonX.Y/site-packages/ — self-contained.
What to Watch Out For
Fresh clone requires task go:init — go.work is gitignored; a fresh clone has no workspace file. Run task go:init once per clone (or once per machine). The status: guard makes it idempotent — it's safe to run every time.
gen/ts/ must be a workspace member, not file: — a file: reference copies the package into node_modules at install time. After buf generate, the copies are stale until pnpm install runs again. Use workspace:* (symlinked, always live).
Docker build context must be repo root — uv sync --package ml resolves workspace deps (libs/shared, gen/python). Build with docker build -f services/ml/Dockerfile . from the repo root. Use .dockerignore to exclude node_modules/, Go source, and the frontend build cache.
GOWORK=off in CI must be explicit — .gitignore prevents committing go.work but doesn't prevent git add -f. Set GOWORK=off in every CI Go step. Don't rely on the file not being present.
go.mod replace directives are required for CI — go.work resolves in-repo deps locally, but GOWORK=off requires each module's go.mod to have replace directives: replace github.com/your-org/beacon/gen/go => ../../gen/go. Without these, CI fails because the module can't find its in-repo dependencies. Run go mod tidy in each module after adding them.
golangci-lint cannot accept {staged_files} across packages — passing files from multiple packages produces "named files must all be in one directory." Run golangci-lint per-module in pre-push, not per-file in pre-commit.
pnpm and uv both need lock enforcement in CI — pnpm install --frozen-lockfile and uv sync --frozen are the equivalent safety checks. Omitting either means CI silently tests different dep versions than developers have locally.
sources lists must include lockfiles — Task caches by hashing declared sources. If uv.lock, pnpm-lock.yaml, or go.sum changes without any source file changing, Task serves a stale cached build. Add lockfiles to every sources list.
task dev log visibility — three servers writing to one terminal is unreadable. Run them in separate terminals or use a process manager (overmind, foreman, or pnpm turbo run dev for TypeScript). task ts:dev, task go:dev, task py:dev as separate commands is the pragmatic answer.
uv members and pnpm coexistence — uv will error if a members glob matches a directory that has no pyproject.toml. Be explicit: list Python-only directories rather than using broad globs like services/*.
mypy in pre-push, not pre-commit — mypy is 10–30s on a real Python codebase. Every commit, on any .py change. It belongs in pre-push alongside buf breaking, not blocking commits.
uv sync --frozen in CI — without --frozen, uv silently re-resolves if uv.lock is stale, meaning CI may test different dep versions than developers have locally.
--no-editable in Python Docker images — editable installs point at source paths in the build context. Use uv sync --package ml --no-editable in Dockerfiles so the .venv is self-contained.
jq is a prerequisite — Taskfile.go.yml uses go work edit -json | jq. Install with brew install jq or apt install jq. Add to your task prereqs check or README.
proto changes cascade + deployment order — buf breaking catches compile-time breakage, but not runtime version skew. In production, always deploy consumers (Python ML, TypeScript frontend) before producers (Go API) when adding fields. Never remove or rename fields in a single deployment — deprecate first, remove in a later release.
lefthook glob_matcher: doublestar — without this line, **/*.py won't match files more than one directory deep. Always verify with lefthook run pre-commit --all-files after first setup.
Generated code is committed — gen/go/, gen/python/, gen/ts/ are in git. This keeps CI simple and makes stub changes reviewable. Proto changes produce large diffs — that's the tradeoff.
Going Further
Dependency updates across three package managers — use Renovate (not Dependabot — Renovate handles pnpm, uv, and Go modules in a single config). For audits: pnpm audit, uv audit, govulncheck ./... per Go module.
Adding a new service — checklist:
- Python service: add to
pyproject.tomlmembers, createpyproject.toml+Taskfile.yml+dir:include, add tolefthook.ymlpaths, add to CIpypaths-filter - Go service: add to
go.work use, add toTaskfile.go.yml initcmd + lint/test, add CI matrix entry - TypeScript package: add to
pnpm-workspace.yaml, add Taskfile include
Cross-language integration tests — unit tests per language are not enough. Add a task test:integration that starts all three services (via Docker Compose or k3s) and runs contract tests against the live stack. Pact works across Go, Python, and TypeScript.
Environment config — one .env at repo root, read by all three languages:
- TypeScript:
dotenvor@t3-oss/env-nextjs - Python:
pydantic-settingswithenv_file = ".env" - Go:
godotenv.Load()or pass via Docker env
CODEOWNERS — proto/ should require review from both Go and Python leads. Use GitHub CODEOWNERS to enforce this on every proto PR.
Hot reload across proto changes — task dev doesn't re-run buf generate when .proto files change. Add a file-watcher task (e.g., via watchexec or task --watch) that runs buf generate and restarts affected dev servers on .proto changes.
Top comments (0)