DEV Community: Lakhan Samani

Your AI assistant can leak files people aren't allowed to see — here's how to fix it

Lakhan Samani — Tue, 23 Jun 2026 22:26:40 +0000

Permission-aware RAG with Authorizer + OpenFGA + Qdrant: a runnable demo where two users ask the same question and get different answers — because the vector search itself respects who can see what. Fully self-hosted.

TL;DR

Vector search ignores your permission model: an engineer can ask your RAG assistant about board-level financials and get them, because semantically close = retrieved, and a restricted chunk in the prompt can't be un-read.

Fix it with a pre-filter: Authorizer's embedded OpenFGA answers list_permissions with the user's token, and that allow-list becomes a Qdrant must payload filter — forbidden chunks are never scored.

Revocation is one tuple deletion, effective on the next question. No re-indexing, no re-login.

Verified live: 63 tests green (including a dedicated fail-closed permission suite), official quay.io/authorizer/authorizer:2.3.0 image, real grant/revoke cycle. Run it: docker compose up -d && python scripts/fga_seed.py && python scripts/fga_demo.py --llm.

There's a quiet security regression happening in almost every company shipping AI features right now.

You spent years getting access control right. Row-level security in Postgres. Folder permissions in Drive. Channel membership in Slack. Then someone embedded all of it into a vector database, pointed an LLM at it, and called it a knowledge assistant.

The assistant doesn't know about your permission model. Similarity search returns whatever is semantically close — the intern asking "how do raises work?" gets chunks of the compensation memo, because cosine similarity has no concept of "need to know." And once a restricted chunk lands in the prompt, it's over: you cannot redact what the model has already read.

The common fix is a post-filter: run the vector search first, then check each result against your permission system and drop what the user shouldn't see. Most off-the-shelf "permission-aware retriever" wrappers work exactly this way. It's better than nothing, but the forbidden chunks were still scored, fetched, and held in your application's memory before being discarded — and if 3 of your top-4 results get dropped, your agent answers from a starved context. When the checks run against a hosted authorization service, you also pay a network round trip per result, and your company's entire who-can-see-what graph lives on someone else's infrastructure.

We built it the other way around — as a pre-filter, fully self-hosted, on an open-source stack — and shipped it as a runnable demo. The demo's knowledge base belongs to a fictional ACME Corp and holds four documents: a public onboarding guide, an engineering tech-stack doc, a finance-only Q4 financial report, and a security policy. Picture the most relatable version of the leak: an engineer asking the assistant about company financials. Same question, two users, different answers. Here's the actual output:

👤 alice@example.com   (engineering)   may view: onboarding_guide.txt, tech_stack.txt
   ❓ What was our Q4 revenue and cash runway?
   💬 I don't have enough information in the knowledge base to answer this question.

👤 carol@example.com   (finance)       may view: financial_report.txt, onboarding_guide.txt
   ❓ What was our Q4 revenue and cash runway?
   💬 The Q4 revenue for ACME Corp was $14.2M, and the monthly cash burn is $1.1M —
      a 26-month runway at current burn.

Alice isn't being told "access denied." ACME's financial report simply does not exist in her world — it was never a search candidate, never retrieved, never in the prompt. The model cannot leak what it never saw. Carol, who's on the finance team, gets the number.

Here's the use case at a glance — the same question, the two outcomes, and the open-source stack underneath:

The stack — open source end to end

Every piece runs on your hardware, and each one earns its place:

Authorizer (≥ v2.3.0) — open-source authentication and authorization in one Go binary. It embeds an OpenFGA engine (the open-source implementation of Google's Zanzibar relationship-based access control), so the server that issues your users' tokens also answers "can this user view this document?" — in-process, no extra service to deploy.
Qdrant — the vector database whose payload filters make true pre-filtering possible: a must condition is evaluated during the HNSW vector search, not bolted on after. This single feature is what lets the permission boundary live inside the retrieval step. Its built-in dashboard (/dashboard) also lets you inspect exactly which chunks and payloads exist.
FastEmbed — local ONNX embeddings (BAAI/bge-small-en-v1.5, 384 dimensions, ~25 MB). No embedding API, no per-token costs, works offline after the first download.
Ollama — the local LLM (llama3.2 in the demo). Prompts containing your internal documents never leave the machine.
Gradio — the demo's web UI, now with a login row so each person chats as themselves.

Nothing in this chain phones home: not the documents, not the tokens, not the permission graph, not the prompts.

docker compose up -d              # Qdrant + Authorizer (quay.io/authorizer/authorizer:2.3.0)
python scripts/fga_seed.py        # demo users, authorization model, grants
python scripts/fga_demo.py --llm  # watch two users get different answers

Permissions are relationships, not roles

The document side of the authorization model is six lines of OpenFGA DSL:

type document
  relations
    define owner: [user]
    define viewer: [user, user:*, team#member]
    define blocked: [user]
    define can_view: (viewer or owner) but not blocked

Access is granted by writing facts, not code:

user:*                    viewer   document:onboarding_guide.txt   # public
team:engineering#member   viewer   document:tech_stack.txt         # a team, not a user list
team:finance#member       viewer   document:financial_report.txt   # finance only
team:security#member      viewer   document:security_policy.txt
user:<alice>              member   team:engineering
user:<carol>              member   team:finance

That team:finance#member viewer tuple is the whole ballgame for our example: Alice is a member of team:engineering, not team:finance, so she has no path to document:financial_report.txt. Carol does.

Notice what's not here: no changes to the ingestion pipeline, no per-user collections, no ACL columns bolted onto chunk metadata. Each Qdrant chunk already carries its source filename in the payload — the FGA object is simply document:<that filename>. The payload you already have is the join key.

How does it work? The architecture

Under the hood it's one Python app process talking to three local servers. The important part is the order: the permission check happens before the vector search, so off-limits files are never even embedded into the candidate set.

It comes down to two API calls.

One: before searching, ask Authorizer what this user may see — authenticated with the user's own token:

allowed = authz.allowed_documents(user_token)
# GraphQL: list_permissions(relation: "can_view", object_type: "document")
# Alice (engineering) → ["onboarding_guide.txt", "tech_stack.txt"]   # no financial_report

Two: hand that allow-list to Qdrant as a must condition:

FieldCondition(key="source", match=MatchAny(any=allowed))

Qdrant evaluates the condition during HNSW traversal. Forbidden vectors are never candidates — top-k stays full of the best chunks the user may see, instead of being decimated by after-the-fact redaction.

And because the permission call carries the end user's JWT, Authorizer pins the subject server-side. An explicit "check as someone else" is rejected unless the caller is a super-admin. A fully prompt-injected agent has nothing to escalate with: it asks as Alice, it gets Alice's answer — and Alice can't see finance.

Fail closed, or don't bother

The demo treats every ambiguous state as deny:

Authorizer unreachable → AuthorizationError, not an unrestricted search.
Authorization configured but no token supplied → error, not a silent bypass.
Permission list truncated → refuse, a partial allow-list is not the full one.
Empty allow-list → refusal answer without ever calling the LLM.
And after generation, the cited sources are batch re-checked with check_permissions — a grant revoked mid-request raises instead of leaking.

That last list is about 30 lines of code total. It's the difference between a security feature and a security theater feature.

Revocation that actually revokes

The demo's second act grants Bob engineering access — one tuple write — and his very next question retrieves the tech-stack doc. Then it deletes the tuple:

⚡ Granting: bob → member → team:engineering
👤 bob   may view: onboarding_guide.txt, tech_stack.txt    ← immediately

⚡ Revoking the same tuple
👤 bob   may view: onboarding_guide.txt                    ← immediately

No re-embedding. No re-indexing. No re-login. No "takes effect after the nightly sync." Permissions live in tuples, not in the vector index — offboarding a contractor is one _fga_delete_tuples call, and their next question is denied.

What we verified (and what we'd tell you anyway)

This isn't a whiteboard architecture. The demo ships a dedicated fail-closed permission suite (authz client, Qdrant pre-filter semantics, pipeline enforcement) — the full suite is 63 tests, all green — and the whole flow ran live against the official quay.io/authorizer/authorizer:2.3.0 image, including the grant/revocation cycle above.

Honest notes from the build: list_permissions caps at 1,000 objects (the client fails closed on truncation — model access via teams/folders to stay under it); contextual tuples work on checks but not on the list call; and Authorizer's CSRF guard requires an Origin header on server-side POSTs — your reverse proxy or HTTP client needs to send one.

FAQ

Do I need to re-index Qdrant when permissions change?

No. Permissions live in OpenFGA relationship tuples, not in the vector index. Granting access is one _fga_write_tuples call and revoking is one _fga_delete_tuples call — both take effect on the user's very next question. The embeddings, payloads, and collections in Qdrant are never touched.

What happens if Authorizer is unreachable mid-request?

The pipeline fails closed: the permission client raises an error and the user gets no documents — never all documents. The same applies to expired tokens, missing tokens when authorization is enabled, and truncated permission lists. Treating every ambiguous state as "deny" is roughly 30 lines of code in the demo.

Can a prompt-injected agent bypass the filter?

No. The agent calls Authorizer with the end user's own JWT, and the server pins the permission subject from that token — a "check as someone else" parameter is rejected unless the caller is a super-admin. A hijacked agent holds no privileged credential, so it asks as the user and gets the user's answer: denied.

Does this work with LangChain or LlamaIndex?

The demo's permission calls go through the official authorizer-py SDK — login, check_permissions, list_permissions, all typed and fail-closed. There's no dedicated LangChain/LlamaIndex retriever wrapper yet, but the pattern (allow-list → vector-store metadata filter) ports directly to any retriever that supports pre-search filtering — Qdrant's LangChain and LlamaIndex integrations both expose payload filters. The demo's authz.py is the thin glue, and packaging it as a reusable retriever is the obvious next step.

What are the current limits?

list_permissions caps at 1,000 objects per call (the client fails closed on truncation — model access via teams or folders to stay under it). Contextual tuples work on check_permissions but not on the list call, parameterized CEL conditions aren't exposed through the API, and the embedded FGA engine needs a SQL datastore.

Run the demo yourself, in four steps

Step 1 — Start the tools. One command brings up Qdrant and Authorizer (with its embedded OpenFGA engine):

docker compose up -d        # Qdrant on :6333, Authorizer on :8080

Step 2 — Seed ACME's users and permissions. The seed script is idempotent: it signs up the three demo users, installs the authorization model, and writes the relationship tuples — all skipped on re-runs.

python scripts/fga_seed.py
# ✓ Created users alice (engineering), carol (finance), bob (new hire)
# ✓ Installed authorization model
# ✓ Granted: team:finance#member viewer document:financial_report.txt …

This produces ACME's access matrix:

document	alice (eng)	bob (new hire)	carol (finance)
`onboarding_guide.txt`	✅	✅	✅
`tech_stack.txt`	✅	❌	❌
`financial_report.txt`	❌	❌	✅
`security_policy.txt`	❌	❌	❌

Step 3 — Log in. Launch the UI; it asks you to log in first (it can't answer anything until it knows who you are), with the demo accounts shown right on the page:

python src/app.py --authorizer http://localhost:8080

Log in as alice@example.com (engineering) or carol@example.com (finance), password Demo@Pass123. The login screen is shown first — the credentials are verified against Authorizer, and the chat only appears once you're authenticated.

Step 4 — Ask the same question as different people. Ask "What was our Q4 revenue?" as Alice and you get "I don't have enough information" — the financial report was never retrieved. Ask Carol and she gets ACME's $14.2M. Prefer the terminal? python scripts/fga_demo.py --llm runs the whole comparison, including a live grant-and-revoke.

Your AI assistant can leak files people aren't allowed to see — here's how to fix it

Lakhan Samani — Tue, 23 Jun 2026 22:26:40 +0000

TL;DR

Vector search ignores your permission model: an engineer can ask your RAG assistant about board-level financials and get them, because semantically close = retrieved, and a restricted chunk in the prompt can't be un-read.

Fix it with a pre-filter: Authorizer's embedded OpenFGA answers list_permissions with the user's token, and that allow-list becomes a Qdrant must payload filter — forbidden chunks are never scored.

Revocation is one tuple deletion, effective on the next question. No re-indexing, no re-login.

Verified live: 63 tests green (including a dedicated fail-closed permission suite), official quay.io/authorizer/authorizer:2.3.0 image, real grant/revoke cycle. Run it: docker compose up -d && python scripts/fga_seed.py && python scripts/fga_demo.py --llm.

There's a quiet security regression happening in almost every company shipping AI features right now.

👤 alice@example.com   (engineering)   may view: onboarding_guide.txt, tech_stack.txt
   ❓ What was our Q4 revenue and cash runway?
   💬 I don't have enough information in the knowledge base to answer this question.

👤 carol@example.com   (finance)       may view: financial_report.txt, onboarding_guide.txt
   ❓ What was our Q4 revenue and cash runway?
   💬 The Q4 revenue for ACME Corp was $14.2M, and the monthly cash burn is $1.1M —
      a 26-month runway at current burn.

Here's the use case at a glance — the same question, the two outcomes, and the open-source stack underneath:

The stack — open source end to end

Every piece runs on your hardware, and each one earns its place:

Authorizer (≥ v2.3.0) — open-source authentication and authorization in one Go binary. It embeds an OpenFGA engine (the open-source implementation of Google's Zanzibar relationship-based access control), so the server that issues your users' tokens also answers "can this user view this document?" — in-process, no extra service to deploy.
Qdrant — the vector database whose payload filters make true pre-filtering possible: a must condition is evaluated during the HNSW vector search, not bolted on after. This single feature is what lets the permission boundary live inside the retrieval step. Its built-in dashboard (/dashboard) also lets you inspect exactly which chunks and payloads exist.
FastEmbed — local ONNX embeddings (BAAI/bge-small-en-v1.5, 384 dimensions, ~25 MB). No embedding API, no per-token costs, works offline after the first download.
Ollama — the local LLM (llama3.2 in the demo). Prompts containing your internal documents never leave the machine.
Gradio — the demo's web UI, now with a login row so each person chats as themselves.

Nothing in this chain phones home: not the documents, not the tokens, not the permission graph, not the prompts.

docker compose up -d              # Qdrant + Authorizer (quay.io/authorizer/authorizer:2.3.0)
python scripts/fga_seed.py        # demo users, authorization model, grants
python scripts/fga_demo.py --llm  # watch two users get different answers

Permissions are relationships, not roles

The document side of the authorization model is six lines of OpenFGA DSL:

type document
  relations
    define owner: [user]
    define viewer: [user, user:*, team#member]
    define blocked: [user]
    define can_view: (viewer or owner) but not blocked

Access is granted by writing facts, not code:

user:*                    viewer   document:onboarding_guide.txt   # public
team:engineering#member   viewer   document:tech_stack.txt         # a team, not a user list
team:finance#member       viewer   document:financial_report.txt   # finance only
team:security#member      viewer   document:security_policy.txt
user:<alice>              member   team:engineering
user:<carol>              member   team:finance

How does it work? The architecture

It comes down to two API calls.

One: before searching, ask Authorizer what this user may see — authenticated with the user's own token:

allowed = authz.allowed_documents(user_token)
# GraphQL: list_permissions(relation: "can_view", object_type: "document")
# Alice (engineering) → ["onboarding_guide.txt", "tech_stack.txt"]   # no financial_report

Two: hand that allow-list to Qdrant as a must condition:

FieldCondition(key="source", match=MatchAny(any=allowed))

Fail closed, or don't bother

The demo treats every ambiguous state as deny:

Authorizer unreachable → AuthorizationError, not an unrestricted search.
Authorization configured but no token supplied → error, not a silent bypass.
Permission list truncated → refuse, a partial allow-list is not the full one.
Empty allow-list → refusal answer without ever calling the LLM.
And after generation, the cited sources are batch re-checked with check_permissions — a grant revoked mid-request raises instead of leaking.

That last list is about 30 lines of code total. It's the difference between a security feature and a security theater feature.

Revocation that actually revokes

The demo's second act grants Bob engineering access — one tuple write — and his very next question retrieves the tech-stack doc. Then it deletes the tuple:

⚡ Granting: bob → member → team:engineering
👤 bob   may view: onboarding_guide.txt, tech_stack.txt    ← immediately

⚡ Revoking the same tuple
👤 bob   may view: onboarding_guide.txt                    ← immediately

What we verified (and what we'd tell you anyway)

FAQ

Do I need to re-index Qdrant when permissions change?

What happens if Authorizer is unreachable mid-request?

Can a prompt-injected agent bypass the filter?

Does this work with LangChain or LlamaIndex?

What are the current limits?

Run the demo yourself, in four steps

Step 1 — Start the tools. One command brings up Qdrant and Authorizer (with its embedded OpenFGA engine):

docker compose up -d        # Qdrant on :6333, Authorizer on :8080

python scripts/fga_seed.py
# ✓ Created users alice (engineering), carol (finance), bob (new hire)
# ✓ Installed authorization model
# ✓ Granted: team:finance#member viewer document:financial_report.txt …

This produces ACME's access matrix:

document	alice (eng)	bob (new hire)	carol (finance)
`onboarding_guide.txt`	✅	✅	✅
`tech_stack.txt`	✅	❌	❌
`financial_report.txt`	❌	❌	✅
`security_policy.txt`	❌	❌	❌

Step 3 — Log in. Launch the UI; it asks you to log in first (it can't answer anything until it knows who you are), with the demo accounts shown right on the page:

python src/app.py --authorizer http://localhost:8080

Log in as alice@example.com (engineering) or carol@example.com (finance), password Demo@Pass123. The login screen is shown first — the credentials are verified against Authorizer, and the chat only appears once you're authenticated.

Authorizer v2 Is Here: Self-Hosted Auth, Rebuilt From the Ground Up

Lakhan Samani — Wed, 01 Apr 2026 12:41:18 +0000

We just shipped Authorizer v2 — a major rewrite of our open-source, self-hosted authentication and authorization server.

If you've ever been frustrated by per-seat auth pricing, vendor lock-in, or shipping your users' data to someone else's cloud — this release is for you.

TL;DR: Single Go binary. 13+ database backends. CLI-driven config. OAuth 2.0/OIDC compliant. Deploy in 5 minutes. Free forever.

GitHub | Docs | Website | Migration Video

Why We Built v2

Authorizer v1 worked. Teams used it in production. But we kept hearing the same feedback:

"Config stored in the database felt fragile."
"I want to manage auth config the same way I manage everything else — through code."
"It's hard to audit what changed and when."

So we rethought the entire configuration model.

v1: Configuration lived in the database, encrypted. You changed settings through the dashboard UI or a GraphQL mutation. Convenient, but opaque — you couldn't version-control your auth config, couldn't audit changes easily, and secrets sat in a persistent store.

v2: All configuration is passed via CLI flags at startup. That's it. No .env files. No database-stored config. No mutation to accidentally expose secrets. Your auth server is configured the same way you configure every other 12-factor service.

authorizer \
  --database-type postgres \
  --database-url "postgres://user:pass@localhost:5432/auth" \
  --jwt-type=HS256 \
  --jwt-secret=test \
  --admin-secret=admin \
  --client-id=123456 \
  --client-secret=secret

That's a production-ready auth server. One command. No magic.

What's in v2

13+ Database Backends

Use whatever database your team already runs:

Category	Supported
SQL	PostgreSQL, MySQL, SQLite, SQL Server, MariaDB, YugabyteDB, PlanetScale, CockroachDB, LibSQL
NoSQL	MongoDB, ArangoDB, CassandraDB, ScyllaDB
Cloud	AWS DynamoDB, Couchbase

No other open-source auth server supports this many backends. If you're already running MongoDB or DynamoDB — you don't need to spin up a separate Postgres just for auth.

11 Social Login Providers

Google, GitHub, Facebook, Apple, LinkedIn, Microsoft, Discord, Twitter, Twitch, Roblox — all configured with a pair of CLI flags:

--google-client-id "..." --google-client-secret "..."
--github-client-id "..." --github-client-secret "..."

Multi-Factor Authentication

TOTP — Google Authenticator, Authy, 1Password
Email OTP — One-time codes via email
SMS OTP — Via Twilio integration
Enforceable globally with --enforce-mfa

Full OAuth 2.0 / OIDC Compliance

Authorization code flow with PKCE (RFC 7636)
Implicit token and ID token flows
JWKS endpoint (/.well-known/jwks.json)
9 JWT signing algorithms (HS256/384/512, RS256/384/512, ES256/384/512)
Custom access token claims via JavaScript scripts

Developer Experience

GraphQL API — Introspectable schema, admin operations prefixed with _
REST endpoints — Standard OAuth 2.0/OIDC paths
SDKs — React, JavaScript, Go, Svelte, Vue, Flutter
Built-in UI — Login/signup pages out of the box, themeable
Admin dashboard — User management, role assignment, email templates
Webhooks — 8 event types for real-time integrations

Role-Based Access Control

Define roles, set defaults, protect sensitive ones:

--roles "user,admin,editor" \
--default-roles "user" \
--protected-roles "admin"

One-Command Deployment

# Docker
docker run -p 8080:8080 -u root \
  -v authorizer_data:/authorizer/data \
  lakhansamani/authorizer \
  --database-type=sqlite \
  --database-url=/authorizer/data/data.db \
  --client-id=123456 \
  --client-secret=secret \
  --admin-secret=admin \
  --jwt-type=HS256 \
  --jwt-secret=test

# Or one-click deploy on Railway, Heroku, Render, Koyeb

Single binary. No JVM. No app server. No runtime dependencies.

What Changed From v1

If you're upgrading, here's what matters:

	v1	v2
Configuration	Stored in DB, editable via dashboard	CLI flags only, immutable at runtime
Binary name	`server`	`authorizer`
Env vars	Read from `.env` and OS	Pass as CLI arguments
Dashboard	Can change server config	Read-only (user management only)
Mobile auth	Separate `mobile_signup`/`mobile_login`	Use `signup`/`login` with `phone_number`
Admin auth	Header always enabled	Can disable header auth for security
SDK versions	authorizer-js v2, authorizer-react v1	authorizer-js v3, authorizer-react v2

We wrote a detailed migration guide covering every breaking change. Prefer video? Here's a step-by-step migration walkthrough on YouTube.

What's Coming Next: The Roadmap

We're not stopping here. Here's what's planned across five phases:

Phase 1: Security Hardening

The foundation for enterprise adoption:

Rate limiting & brute force protection — Per-IP, per-user throttling and account lockout
CAPTCHA integration — Cloudflare Turnstile and Google reCAPTCHA v3
Leaked password detection — Have I Been Pwned API integration
Structured audit logs — Queryable event trail for compliance
Prometheus metrics — /metrics endpoint for observability
Session security — Device fingerprinting, unrecognized device alerts, remote revocation

Phase 2: Authorization & Machine-to-Machine

Moving beyond basic RBAC:

Fine-grained permissions — Resource-level access control (document:read, project:admin)
M2M authentication — OAuth 2.0 client credentials grant for service-to-service
Service accounts — Application identities that aren't tied to humans
API key management — Let your users create and manage their own API keys
Organization enhancements — Domain-based routing, org-level policies, invitations

Phase 3: Enterprise SSO & Federation

What enterprise buyers ask for on day one:

SAML 2.0 — Connect to Okta, Azure AD, OneLogin
SCIM 2.0 / Directory Sync — Automated user provisioning and deprovisioning
Authorizer as OIDC Provider — Issue tokens for downstream services
Self-service admin portal — Let customer IT teams configure their own SSO

Phase 4: AI-Era Authentication

Auth is changing. AI agents need identity too:

MCP (Model Context Protocol) authorization — Secure tool access for AI agents
Agent-to-Agent (A2A) authentication — Identity and delegation for autonomous agents
OAuth 2.1 compliance — Mandatory PKCE, no implicit grant, refresh token rotation
Token exchange (RFC 8693) — Delegation and impersonation flows
Dynamic client registration (RFC 7591) — Programmatic OAuth client creation

Phase 5: Advanced Security & Modern Standards (Q2-Q3 2027)

Passkeys / WebAuthn (FIDO2) — Passwordless with hardware keys
DPoP (RFC 9449) — Proof-of-possession tokens to prevent token theft
Advanced bot protection — Risk scoring, credential stuffing detection
SIEM integration — Stream logs to Datadog, Splunk, Elastic

The full roadmap is on GitHub.

Why Self-Hosted Auth Matters in 2026

Three trends are making self-hosted auth more relevant than ever:

1. Data sovereignty isn't optional anymore. GDPR enforcement is accelerating. New regulations in India, Brazil, and across APAC require data residency. If your auth provider stores user data in a region you can't control, you have a compliance problem.

2. Auth pricing doesn't scale. Hosted auth providers get expensive fast — they charge per user, per connection, or per feature. With Authorizer, you pay for a server. That's it.

3. AI agents need auth too. MCP, A2A, and OAuth 2.1 are the emerging standards for agent authentication. The auth layer needs to evolve — and you want that evolution to happen on infrastructure you control.

Get Started

5-minute quickstart:

docker run -p 8080:8080 -u root \
  -v authorizer_data:/authorizer/data \
  lakhansamani/authorizer \
  --database-type=sqlite \
  --database-url=/authorizer/data/data.db \
  --client-id=123456 \
  --client-secret=secret \
  --admin-secret=admin \
  --jwt-type=HS256 \
  --jwt-secret=test

Open http://localhost:8080 — you have a working auth server with a login page.

Add it to your React app:

npm install @authorizerdev/authorizer-react

import { AuthorizerProvider, Authorizer } from '@authorizerdev/authorizer-react';

function App() {
  return (
    <AuthorizerProvider
      config={{
        authorizerURL: 'http://localhost:8080',
        redirectURL: window.location.origin,
        clientID: 'my-app',
      }}
    >
      <Authorizer />
    </AuthorizerProvider>
  );
}

That's email/password auth, social logins, and session management — in 15 lines.

One-click cloud deploy:

Join the Community

Star us on GitHub: github.com/authorizerdev/authorizer
Website: authorizer.dev
Read the docs: docs.authorizer.dev
Watch the migration video: YouTube — v1 to v2 migration
Join Discord: discord.gg/n7DfTjCAn — Chat with the team and other developers
Contribute: Check out our contributing guide

Authorizer is Apache 2.0 licensed. It's free, it's open source, and your data stays yours.

We'd love your feedback, bug reports, and contributions. If this solves a problem for you — give us a star. It helps more than you think.

Sponsor Authorizer

Authorizer is built and maintained by the community. If it saves you time or money, consider sponsoring the project to keep development going:

Sponsor Authorizer on GitHub

Built with Go. Powered by the community. Owned by you.

About the author: I'm Lakhan Samani, creator of Authorizer. Connect with me on LinkedIn or X/Twitter.

Deploying Our Distributed System on Kubernetes with Kind - Part 7

Lakhan Samani — Mon, 17 Mar 2025 09:59:26 +0000

In the previous blog posts, we designed and implemented a distributed system comprising User Service (users) and Order Service (ordered) with observability using OpenTelemetry, Jaeger, Prometheus, and Alertmanager. In this final part, we'll deploy the entire system on Kubernetes (Kind) and demonstrate how it benefits distributed systems.

Why Deploy on Kubernetes?

Deploying our distributed system on Kubernetes offers several benefits:

Scalability: Easily scale services as traffic grows.
Fault Tolerance: Kubernetes ensures services restart on failure.
Observability: Seamless monitoring using Prometheus and Jaeger.
Portability: Deploy the same stack in any cloud or local environment.

Setting Up the Kind Cluster

We'll use Kind (Kubernetes in Docker) to set up a local Kubernetes cluster.

Step 1: Install Kind & kubectl

Ensure Kind and kubectl are installed:

# Install Kind
curl -Lo ./kind https://kind.sigs.k8s.io/dl/v0.20.0/kind-linux-amd64
chmod +x kind
mv kind /usr/local/bin/

# Install kubectl
curl -LO "https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl"
chmod +x kubectl
mv kubectl /usr/local/bin/

Step 2: Create a Kind Cluster

kind create cluster --name distributed-system --config kind.yaml

kind.yaml:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
  - role: control-plane
    extraPortMappings:
      - containerPort: 5432 # postgres
        hostPort: 5432
      - containerPort: 50051 # userd grpc
        hostPort: 50051
      - containerPort: 50052 # orderd grpc
        hostPort: 50052
      - containerPort: 9091 # userd metrics
        hostPort: 9091
      - containerPort: 9092 # orderd metrics
        hostPort: 9092
      - containerPort: 16686 # jaeger ui
        hostPort: 16686
      - containerPort: 4317 # jaeger grpc
        hostPort: 4317
      - containerPort: 4318 # jaeger http
        hostPort: 4318
      - containerPort: 9090 # prometheus
        hostPort: 9090
      - containerPort: 9093 # alertmanager
        hostPort: 9093

Deploying Databases (PostgreSQL)

We need userdb and orderdb for our services.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: postgres
spec:
  replicas: 1
  selector:
    matchLabels:
      app: postgres
  template:
    metadata:
      labels:
        app: postgres
    spec:
      containers:
        - name: postgres
          image: postgres:latest
          env:
            - name: POSTGRES_USER
              value: postgres
            - name: POSTGRES_PASSWORD
              value: postgres
          ports:
            - containerPort: 5432
---
apiVersion: v1
kind: Service
metadata:
  name: postgres
spec:
  ports:
    - protocol: TCP
      port: 5432
      targetPort: 5432
  selector:
    app: postgres
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: userdb-pvc
spec:
  accessModes:
    - ReadWriteOnce
  resources:
    requests:
      storage: 1Gi
---
apiVersion: batch/v1
kind: Job
metadata:
  name: create-userdb
spec:
  template:
    spec:
      containers:
        - name: create-userdb
          image: postgres:latest
          command: ["/bin/sh", "-c", "psql -U postgres -h postgres -c 'CREATE DATABASE userdb;'" ]
          env:
            - name: PGPASSWORD
              value: postgres
      restartPolicy: Never
---
apiVersion: batch/v1
kind: Job
metadata:
  name: create-orderdb
spec:
  template:
    spec:
      containers:
        - name: create-orderdb
          image: postgres:latest
          command: ["/bin/sh", "-c", "psql -U postgres -h postgres -c 'CREATE DATABASE orderdb;'" ]
          env:
            - name: PGPASSWORD
              value: postgres
      restartPolicy: Never

Apply the database setup:

kubectl apply -f postgres.yaml

Deploying the User and Order Services

Each service will be exposed via a Kubernetes Service.

userd Deployment & Service `userd.yaml`

Before deploying this service, we have created docker image and pushed to docker hub

# build image
docker build --platform linux/amd64 -t lakhansamani/ecom-grpc-userd:0.1.0 .

#push image
docker push lakhansamani/ecom-grpc-userd:0.1.0

Note you can use your docker hub account or any other container registry to push image.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: userd
spec:
  replicas: 1
  selector:
    matchLabels:
      app: userd
  template:
    metadata:
      labels:
        app: userd
    spec:
      containers:
        - name: userd
          image: lakhansamani/ecom-grpc-userd:0.1.0
          env:
            - name: DB_URL
              value: "postgres://postgres:postgres@postgres:5432/userdb"
            - name: JWT_SECRET
              value: "secret"
            - name: JAEGER_URL
              value: "jaeger:4317"
          ports:
            - containerPort: 50051
            - containerPort: 9091
---
apiVersion: v1
kind: Service
metadata:
  name: userd
spec:
  ports:
    - name: grpc
      port: 50051
      targetPort: 50051
    - name: metrics
      port: 9091
      targetPort: 9091
  selector:
    app: userd

Apply the deployment:

kubectl apply -f userd.yaml

orderd Deployment & Service `orderd.yaml`

Before deploying this service, we have created docker image and pushed to docker hub

# build image
docker build --platform linux/amd64 -t lakhansamani/ecom-grpc-orderd:0.1.0 .

#push image
docker push lakhansamani/ecom-grpc-orderd:0.1.0

Note you can use your docker hub account or any other container registry to push image.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orderd
spec:
  replicas: 1
  selector:
    matchLabels:
      app: orderd
  template:
    metadata:
      labels:
        app: orderd
    spec:
      containers:
        - name: orderd
          image: lakhansamani/ecom-grpc-orderd:0.1.0
          env:
            - name: DB_URL
              value: "postgres://postgres:postgres@postgres:5432/orderdb"
            - name: USER_SERVICE_URL
              value: "userd:50051"
            - name: JAEGER_URL
              value: "jaeger:4317"
          ports:
            - containerPort: 50052
            - containerPort: 9092
---
apiVersion: v1
kind: Service
metadata:
  name: orderd
spec:
  ports:
    - name: grpc
      port: 50052
      targetPort: 50052
    - name: metrics
      port: 9092
      targetPort: 9092
  selector:
    app: orderd

Apply the file:

kubectl apply -f orderd.yaml

Deploying Observability Stack

Jaeger (Tracing) `jaeger.yaml`

apiVersion: apps/v1
kind: Deployment
metadata:
  name: jaeger
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jaeger
  template:
    metadata:
      labels:
        app: jaeger
    spec:
      containers:
        - name: jaeger
          image: jaegertracing/all-in-one:latest
          ports:
            - containerPort: 16686
            - containerPort: 4317
            - containerPort: 4318
          env:
            - name: COLLECTOR_OTLP_ENABLED
              value: "true"
---
apiVersion: v1
kind: Service
metadata:
  name: jaeger
spec:
  selector:
    app: jaeger
  ports:
    - name: ui
      port: 16686
      targetPort: 16686
    - name: grpc
      port: 4317
      targetPort: 4317
    - name: http
      port: 4318
      targetPort: 4318

kubectl apply -f jaeger.yaml

Prometheus (Metrics) `prometheus.yaml`

apiVersion: v1
kind: ConfigMap
metadata:
  name: prometheus-config
data:
  prometheus.yml: |
    global:
      scrape_interval: 5s

    alerting:
      alertmanagers:
        - static_configs:
            - targets: ["alertmanager:9093"]

    rule_files:
      - alert_rules.yaml

    scrape_configs:
      - job_name: "userd"
        static_configs:
          - targets: ["userd:9091"]

      - job_name: "orderd"
        static_configs:
          - targets: ["orderd:9092"]

  alert_rules.yaml: |
    groups:
      - name: userd
        rules:
          - alert: HighInvalidLoginAttempts
            expr: increase(userd_service_login_handler{result="invalid_password"}[1m]) > 3
            for: 1m
            labels:
              severity: warning
            annotations:
              summary: "High Invalid Login Attempts"
              description: "More than 3 invalid login attempts detected in the last minute."
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: prometheus
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      containers:
        - name: prometheus
          image: prom/prometheus:v2.37.0
          args:
            - "--config.file=/etc/prometheus/prometheus.yml"
            - "--storage.tsdb.path=/prometheus"
            - "--web.enable-lifecycle"
          ports:
            - name: http
              containerPort: 9090
          volumeMounts:
            - name: prometheus-config-volume
              mountPath: /etc/prometheus/prometheus.yml
              subPath: prometheus.yml
            - name: prometheus-config-volume
              mountPath: /etc/prometheus/alert_rules.yaml
              subPath: alert_rules.yaml
      volumes:
        - name: prometheus-config-volume
          configMap:
            name: prometheus-config
---
apiVersion: v1
kind: Service
metadata:
  name: prometheus
spec:
  selector:
    app: prometheus
  ports:
    - name: http
      port: 9090
      targetPort: 9090

kubectl apply -f prometheus.yaml

Alertmanager (Alerts) `alertmanager.yaml`

apiVersion: v1
kind: ConfigMap
metadata:
  name: alertmanager-config
data:
  alertmanager.yml: |
    global:
      resolve_timeout: 5m

    route:
      receiver: "slack-notifications"

    receivers:
      - name: "slack-notifications"
        slack_configs:
          - channel: "#alerts"
            send_resolved: true
            username: "prometheus"
            api_url: "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: alertmanager
spec:
  replicas: 1
  selector:
    matchLabels:
      app: alertmanager
  template:
    metadata:
      labels:
        app: alertmanager
    spec:
      containers:
        - name: alertmanager
          image: prom/alertmanager:v0.25.0
          args:
            - "--config.file=/etc/alertmanager/alertmanager.yml"
          ports:
            - containerPort: 9093
          volumeMounts:
            - name: config-volume
              mountPath: /etc/alertmanager
      volumes:
        - name: config-volume
          configMap:
            name: alertmanager-config
---
apiVersion: v1
kind: Service
metadata:
  name: alertmanager
spec:
  selector:
    app: alertmanager
  ports:
    - name: http
      port: 9093
      targetPort: 9093

kubectl apply -f alertmanager.yaml

Accessing Services

After deployment, access services on localhost:

User Service: grpcurl -plaintext localhost:50051 list
Order Service: grpcurl -plaintext localhost:50052 list
Jaeger UI: http://localhost:16686
Prometheus: http://localhost:9090
Alertmanager: http://localhost:9093

Note: if it does not work, try port forwarding those port to local ports. eg kubectl port-forward svc/jaeger 16686:16686

Repository

All configuration files has been published here:

Github Repo

Conclusion

With Kubernetes and Kind, we've successfully deployed our distributed system with observability and monitoring. This setup provides scalability, resilience, and detailed insights into system health. From here, you can:

Scale the services using kubectl scale deployment userd --replicas=3
Deploy to the cloud using Kubernetes clusters like GKE or EKS.

This marks the end of our distributed system journey. Happy coding! 🚀

Implementing monitoring and alerting for distributed system - Part 6

Lakhan Samani — Tue, 11 Mar 2025 08:25:47 +0000

Introduction

In Part 5, we implemented the tracing in userd & orderd services. In a distributed system, tracing helps follow a request's journey across services, while metrics play a crucial role in observability by providing visibility into system behavior, performance, and health. In distributed systems, they help with:

Performance Monitoring – Detect bottlenecks and optimize services.
Latency Tracking – Identify slow services in request chains.
Error Detection – Spot failing components before they cause outages.
Capacity Planning – Predict resource needs to avoid over/under-provisioning.
Alerting & Troubleshooting – Enable proactive issue resolution.

In this post, we will set up metrics and alerting in our e-commerce system userd and orderd services using Prometheus

Why Prometheus?

Prometheus is one of the best tools for monitoring gRPC-based distributed systems because of its scalability, pull-based model, efficient storage, and powerful query language (PromQL). Here’s why it’s widely used:

Pull-Based Model for Scalability
Efficient Storage & Retrieval
gRPC-Native Support
Powerful Query Language (PromQL)
Easy Integration with Alerting & Dashboards

What kind of metrics are we interested in?

Request Metrics
- Total Requests (grpc_server_requests_total): Number of gRPC requests received.
- Request Rate: Requests per second (RPS).
Latency Metrics
- Request Duration (grpc_server_handling_seconds): Time taken to handle each request.
Error Metrics
- gRPC Status Codes (grpc_server_responses_total with labels for status_code)
Custom Business Metrics
- Login API failure / success counter
- Invalid login attempts counter
- Register API failure / success counter
- Me API failure / success counter
- Create Order API failure / success counter
- Fetch Order API failure / success counter

Integrating Metrics in userd

We updated userd to start an HTTP metrics server that Prometheus can scrape.

Step 1: Configure Prometheus Go Client

We will be using prometheus client to start metrics server as a go routine that can run in background.

This endpoint is then scraped by prometheus.

We will be using go-grpc-middleware for registering the standard metrics of grpc server.


import (
    "github.com/prometheus/client_golang/prometheus/promhttp"
    grpcprom "github.com/grpc-ecosystem/go-grpc-middleware/providers/prometheus"
)

const (
    metricsPort = ":9092"
)

var (
    grpcMetrics       = grpcprom.NewServerMetrics()
)

func main() {
    ....
    // Create a new gRPC server
    server := grpc.NewServer(
        grpc.StatsHandler(serverHandler),
        grpc.ChainUnaryInterceptor(
            grpcMetrics.UnaryServerInterceptor(),
        ),
        grpc.ChainStreamInterceptor(
            grpcMetrics.StreamServerInterceptor(),
        ),
    )

    // Start Prometheus HTTP server
    go func() {
        http.Handle("/metrics", promhttp.Handler())
        log.Println("Prometheus metrics server running on port", metricsPort)
        log.Fatal(http.ListenAndServe(metricsPort, nil))
    }()
}

Note we will have similar change in orderd service for server setting in main.go file

Step 2: Define custom metrics for API service for `userd` service

package service

import "github.com/prometheus/client_golang/prometheus"

const (
    serviceName = "userd"
    component   = "service"

    loginHandlerLabel    = "login_handler"
    registerHandlerLabel = "register_handler"
    meHandlerLabel       = "me_handler"

    successResultLabel         = "success"
    userNotFoundResultLabel    = "user_not_found"
    invalidPasswordResultLabel = "invalid_password"
    userExistsResultLabel      = "user_exists"
    invalidTokenResultLabel    = "invalid_token"
    missingTokenResultLabel    = "missing_token"
)

var (
    // Metrics for login API

    loginMetrics = prometheus.NewCounterVec(prometheus.CounterOpts{
        Namespace: serviceName,
        Subsystem: component,
        Name:      loginHandlerLabel,
    }, []string{"result"})

    registerMetrics = prometheus.NewCounterVec(prometheus.CounterOpts{
        Namespace: serviceName,
        Subsystem: component,
        Name:      registerHandlerLabel,
    }, []string{"result"})

    meMetrics = prometheus.NewCounterVec(prometheus.CounterOpts{
        Namespace: serviceName,
        Subsystem: component,
        Name:      meHandlerLabel,
    }, []string{"result"})
)

Step 3: Call metrics in service handlers

Example: Login service/login.go

// Get user by email
resUser, err := s.DBProvider.GetUserByEmail(ctx, email)
if err != nil {
    if errors.Is(err, gorm.ErrRecordNotFound) {
        loginMetrics.WithLabelValues(userNotFoundResultLabel).Inc()
        return nil, errors.New("user not found")
    }
    return nil, err
}

// Match password
if err := bcrypt.CompareHashAndPassword([]byte(resUser.Password), []byte(password)); err != nil {
    loginMetrics.WithLabelValues(invalidPasswordResultLabel).Inc()
    return nil, errors.New("invalid password")
}

// Generate JWT token
token, err := utils.GenerateJWT(s.JWTSecret, resUser.ID)
if err != nil {
    return nil, err
}
loginMetrics.WithLabelValues(successResultLabel).Inc()

Integrating Metrics in orderd

orderd needs both server-side and client-side metrics calls for grpc userd.

Step 1 remains same as `userd` service.

Step 2: Add gRPC Client Interceptors

gRPC client interceptor helps in collecting grpc metrics while making call to userd service here.

var (
    grpcClientMetrics = grpcprom.NewClientMetrics()
)

// Create UserServiceClient using grpc
grpcConn, err := grpc.NewClient(
    userServiceURL,
    grpc.WithTransportCredentials(insecure.NewCredentials()),
    grpc.WithStatsHandler(openTelemetryClientHandler),
    // Add gRPC Client Interceptors for Prometheus Metrics
    grpc.WithUnaryInterceptor(grpcClientMetrics.UnaryClientInterceptor()),
    grpc.WithStreamInterceptor(grpcClientMetrics.StreamClientInterceptor()),
)
userServiceClient := user.NewUserServiceClient(grpcConn)

Step 3: Define metrics for `orderd` services

package service

import "github.com/prometheus/client_golang/prometheus"

const (
    serviceName = "orderd"
    component   = "service"

    totalOrdersCreatedMetric = "total_orders"
    totalFetchedOrdersMetric = "total_fetched_orders"

    successResultLabel = "success"
    failedResultLabel  = "failed"
)

var (
    ordersCreatedMetrics = prometheus.NewCounterVec(prometheus.CounterOpts{
        Namespace: serviceName,
        Subsystem: component,
        Name:      totalOrdersCreatedMetric,
    }, []string{"result"})

    ordersFetchedMetrics = prometheus.NewCounterVec(prometheus.CounterOpts{
        Namespace: serviceName,
        Subsystem: component,
        Name:      totalFetchedOrdersMetric,
    }, []string{"result", "order_id"})
)

Step 4: Example integration of metrics in api services

resOrder, err := s.DBProvider.GetOrderById(ctx, orderID)
if err != nil {
    ordersFetchedMetrics.WithLabelValues(failedResultLabel, orderID).Inc()
    return nil, err
}

ordersFetchedMetrics.WithLabelValues(successResultLabel, orderID).Inc()

Setting Up Prometheus

Step 1: Define Prometheus configurations

Create a file called prometheus.yml with following configurations

global:
  scrape_interval: 5s

scrape_configs:
  - job_name: "userd"
    static_configs:
      - targets: ["host.docker.internal:9091"]

  - job_name: "orderd"
    static_configs:
      - targets: ["host.docker.internal:9092"]

Here we are configuring targets that prometheus will scrape for metrics, note we are using host.docker.internal as we are going to setup with docker, if service and prometheus are on same network you can also use localhost there.

Step 2: Run prometheus using docker

docker run -d --name=prometheus -p 9090:9090 -v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml prom/prometheus

Note: we are using the path for above defined prometheus.yml here.

Open http://localhost:9090 in your browser to visualize metrics.

Setting up Alerts using prometheus

Lets try to setup alert for invalid password attempts > 3

Step 1: Create `alert_rules.yml`

groups:
  - name: grpc_alerts
    rules:
      - alert: HighGRPCFailures
        expr: increase(grpc_client_failed_requests_total[1m]) > 5
        for: 1m
        labels:
          severity: critical
        annotations:
          summary: "High gRPC Client Failures"
          description: "More than 5 gRPC client failures detected in the last minute."

Step 2: Create `alertmanager.yml`

Just a dummy example to setup slack alerts

global:
  resolve_timeout: 5m

route:
  receiver: "slack-notifications"

receivers:
  - name: "slack-notifications"
    slack_configs:
      - channel: "#alerts"
        send_resolved: true
        username: "prometheus"
        api_url: "https://hooks.slack.com/services/YOUR/SLACK/WEBHOOK"

Step 3: Update `prometheus.yml`

global:
  scrape_interval: 5s

alerting:
  alertmanagers:
    - static_configs:
        - targets: ["host.docker.internal:9093"]

rule_files:
  - alert_rules.yml

scrape_configs:
  - job_name: "userd"
    static_configs:
      - targets: ["host.docker.internal:9091"]

  - job_name: "orderd"
    static_configs:
      - targets: ["host.docker.internal:9092"]

Step 4: Start `alertmanager` & Restart `prometheus`

docker run -d --name=alertmanager \
  -p 9093:9093 \
  -v $(pwd)/alertmanager.yml:/etc/alertmanager/alertmanager.yml \
  prom/alertmanager

docker run -d --name=prometheus -p 9090:9090 -v $(pwd)/prometheus.yml:/etc/prometheus/prometheus.yml -v $(pwd)/alert_rules.yml:/etc/prometheus/alert_rules.yml prom/prometheus

Code Links

For complete change log, pls check

Conclusion

We successfully added monitoring and alerting using prometheus metrics and alertmanager to our distributed app

Next Steps:

Explore deploying distributed app on k8s

Stay tuned for the next post! 🚀

Implementing tracing in distributed systems - Part 5

Lakhan Samani — Fri, 07 Mar 2025 18:11:09 +0000

Introduction

In Part 4, we implemented the orderd service. In a distributed system, tracking what happens across multiple services is crucial. Logging provides insights into what occurred, while tracing helps follow a request's journey across services.

In this post, we will set up distributed tracing in our e-commerce system userd and orderd services using OpenTelemetry and Jaeger. This will help us visualize the flow of requests and debug performance bottlenecks.

Check when get order service is called how the trace is made to user service and db methods.

Why OpenTelemetry and Jaeger?

OpenTelemetry (OTel) is a vendor-neutral observability framework that provides standardized tools for metrics, logs, and traces. Jaeger, originally developed at Uber, is a distributed tracing tool that helps monitor and troubleshoot transactions in microservices-based systems.

Why not just logs?

Logs capture discrete events but do not provide a structured way to see how a request moves across multiple services.
Tracing provides an end-to-end view of a request and shows where delays occur.

Benefits of OpenTelemetry and Jaeger:

✅ End-to-end tracing: Tracks requests across microservices.
✅ Root cause analysis: Helps find performance bottlenecks.
✅ Open standard: Works with multiple backends like Jaeger, Zipkin, Prometheus, and Datadog.
✅ Automatic instrumentation: Hooks into gRPC, HTTP, and database calls without major code changes.

Implementing Tracing in userd

We update userd to send traces using OpenTelemetry.

Step 1: Configure OpenTelemetry in main.go

Check the exporter here, we will exporting trace to Jaeger endpoint that way it can collect all the traces.

ctx := context.Background()
exporter, err := otlptracegrpc.New(ctx, otlptracegrpc.WithInsecure(), otlptracegrpc.WithEndpoint("localhost:4317"))
if err != nil {
    log.Fatalf("failed to create OTLP trace exporter: %v", err)
}

tracerProvider := trace.NewTracerProvider(
    trace.WithBatcher(exporter),
    trace.WithResource(resource.NewWithAttributes(
        semconv.SchemaURL,
        semconv.ServiceNameKey.String("userd"),
    )),
)

otel.SetTracerProvider(tracerProvider)

Step 2: Add Tracing to gRPC Server

serverHandler := otelgrpc.NewServerHandler(
    otelgrpc.WithTracerProvider(tracerProvider),
)
server := grpc.NewServer(
    grpc.StatsHandler(serverHandler),
)

Step 3: Add Trace Spans in UserService methods

example, adding trace span to Register service in service/register.go

ctx, span := s.trace.Start(ctx, "Register")
defer span.End()

Step 4: Add trace to db calls.

In order to achieve this, we need to add plugin and call db methods with context

Updated db/db.go db provider

// New creates new database provider
// connects to db and returns the provider
func New(dbURL string) Provider {
    db, err := gorm.Open(postgres.Open(dbURL), &gorm.Config{})
    if err != nil {
        log.Fatalf("Failed to connect to database: %v", err)
    }

    if err := db.Use(tracing.NewPlugin()); err != nil {
        log.Fatalf("Failed to use tracing plugin: %v", err)
    }
    // Auto-migrate User model
    db.AutoMigrate(&User{})

    return &provider{db}
}

Implementing Tracing in orderd

orderd needs both server-side and client-side tracing since it calls userd.

Step 1,2,3,4 remains same as userd service.

Step 5: Add gRPC Client Tracing for userd

openTelemetryClientHandler := otelgrpc.NewClientHandler(
    otelgrpc.WithTracerProvider(tracerProvider),
)
grpcConn, err := grpc.NewClient(
    userServiceURL,
    grpc.WithTransportCredentials(insecure.NewCredentials()),
    grpc.WithStatsHandler(openTelemetryClientHandler),
)
userServiceClient := user.NewUserServiceClient(grpcConn)

Setting Up Jaeger

Step 1: Run Jaeger using Docker

docker run -d --name jaeger \
  -e COLLECTOR_OTLP_ENABLED=true \
  -p 16686:16686 \
  -p 4317:4317 \
  -p 4318:4318 \
  jaegertracing/all-in-one:latest

Step 2: View Traces in Jaeger UI

Open http://localhost:16686 in your browser to visualize traces.

Code Links

For complete change log, pls check

Conclusion

We successfully added OpenTelemetry tracing to our userd and orderd services. By using Jaeger, we can visualize request flows, diagnose bottlenecks, and improve performance in our distributed system.

Next Steps:

Explore monitoring and metrics collection using Prometheus.

Stay tuned for the next post! 🚀

Implementing Order Service with Golang, gRPC and PostgreSQL - Part 4

Lakhan Samani — Thu, 06 Mar 2025 15:22:44 +0000

Introduction

In Part 3 we implemented user service. In this post, we will implement an Order Service using Golang, gRPC, PostgreSQL, and GORM.

Note: We are using static price here for product passed in create order request and we will be connecting to user service via grpc to authorize user.

Project Structure

The folder structure for the service is as follows:

ecom-grpc/orderd/
│-- db/
│   │-- db.go
│   │-- order.go
│-- service/
│   │-- service.go
│   │-- create_order.go
│   │-- get_order.go
│-- main.go
│-- .env
│-- Dockerfile
│-- .dockerignore

Each file serves a specific purpose in maintaining a clean and modular design.

Setting Up the Database

We'll use PostgreSQL as our database. If you don’t have PostgreSQL installed, you can run it using Docker:

docker run --name postgres-cluster -e POSTGRES_PASSWORD=postgres -p 5432:5432 -d postgres
docker exec -it postgres-cluster psql -U postgres -c "CREATE DATABASE orderdb;"

Database Layer (`db/`)

The db package handles all interactions with PostgreSQL.

`db/db.go` - Database Connection

package db

import (
    "log"

    "gorm.io/driver/postgres"
    "gorm.io/gorm"
)

// Provider defines the interface for the database provider
type Provider interface {
    CreateOrder(order *Order) (*Order, error)
    GetOrderById(id string) (*Order, error)
}

// provider implements the Provider interface
type provider struct {
    db *gorm.DB
}

// New creates new database provider
// connects to db and returns the provider
func New(dbURL string) Provider {
    db, err := gorm.Open(postgres.Open(dbURL), &gorm.Config{})
    if err != nil {
        log.Fatalf("Failed to connect to database: %v", err)
    }

    // Auto-migrate User model
    db.AutoMigrate(&Order{})

    return &provider{db}
}

`db/order.go` - Order Model

package db

import (
    "github.com/google/uuid"
    "gorm.io/gorm"

    order "github.com/lakhansamani/ecom-grpc-apis/order/v1"
)

// Order represents the Order model in DB
type Order struct {
    ID        string `gorm:"primaryKey"`
    UserID    string
    Product   string
    Quantity  int32
    UnitPrice float64
}

// AsAPIOrder converts the Order model to API Order
func (o *Order) AsAPIOrder() *order.Order {
    return &order.Order{
        Id:        o.ID,
        UserId:    o.UserID,
        Product:   o.Product,
        Quantity:  o.Quantity,
        UnitPrice: o.UnitPrice,
    }
}

// Before create
func (o *Order) BeforeCreate(tx *gorm.DB) (err error) {
    // Generate UUID
    o.ID = uuid.NewString()
    return
}

// CreateOrder creates a new order in the database
func (p *provider) CreateOrder(o *Order) (*Order, error) {
    err := p.db.Create(o).Error
    return o, err
}

// GetOrderById fetches a order by ID from the database
func (p *provider) GetOrderById(id string) (*Order, error) {
    var o Order
    err := p.db.Where("id = ?", id).First(&o).Error
    return &o, err
}

Service Layer (`service/`)

This layer implements the gRPC server and business logic.

`service/service.go` - Service Dependencies

package service

import (
    "context"
    "errors"

    "google.golang.org/grpc/metadata"

    order "github.com/lakhansamani/ecom-grpc-apis/order/v1"
    user "github.com/lakhansamani/ecom-grpc-apis/user/v1"

    "github.com/lakhansamani/ecom-grpc-orderd/db"
)

type Config struct {
    UserServiceAddress string
}

type Dependencies struct {
    // Add dependencies here
    DBProvider db.Provider
    // UserService user.Service
    UserService user.UserServiceClient
}

// Service implements the Order service.
type Service interface {
    order.OrderServiceServer
}

type service struct {
    Config
    Dependencies
}

// New creates a new Order service.
func New(cfg Config, deps Dependencies) Service {
    return &service{
        Config:       cfg,
        Dependencies: deps,
    }
}

// authorize verifies user using the user service and gets userID
func (s *service) authorize(ctx context.Context) (string, error) {
    md, ok := metadata.FromIncomingContext(ctx)
    if !ok {
        return "", errors.New("missing metadata")
    }

    authHeader, exists := md["authorization"]
    if !exists || len(authHeader) == 0 {
        return "", errors.New("missing authorization token")
    }
    token := authHeader[0]

    // add token to outgoing context
    ctx = metadata.AppendToOutgoingContext(ctx, "authorization", token)

    // Call user service to get user
    userResp, err := s.UserService.Me(ctx, &user.MeRequest{})
    if err != nil {
        return "", err
    }
    return userResp.GetUser().GetId(), nil
}

`service/create_order.go` - Create Order API

package service

import (
    "context"
    "errors"

    order "github.com/lakhansamani/ecom-grpc-apis/order/v1"
    "github.com/lakhansamani/ecom-grpc-orderd/db"
)

// CreateOrder API to create a new order
// Permission: authenticated user
func (s *service) CreateOrder(ctx context.Context, req *order.CreateOrderRequest) (*order.CreateOrderResponse, error) {
    // Authorizer user
    userID, err := s.authorize(ctx)
    if err != nil {
        return nil, err
    }
    // Validate request
    product := req.GetProduct()
    quantity := req.GetQuantity()
    if product == "" {
        return nil, errors.New("product is required")
    }
    if quantity <= 0 {
        return nil, errors.New("quantity should be greater than 0")
    }
    // Static Price
    price := float64(10.5)
    // Save order to database
    resOrder, err := s.DBProvider.CreateOrder(&db.Order{
        UserID:    userID,
        Product:   product,
        Quantity:  quantity,
        UnitPrice: price,
    })
    if err != nil {
        return nil, err
    }
    return &order.CreateOrderResponse{
        Order: resOrder.AsAPIOrder(),
    }, nil
}

`service/get_order.go` - Get Order API

package service

import (
    "context"
    "errors"

    order "github.com/lakhansamani/ecom-grpc-apis/order/v1"
)

// GetOrder API to get order details
// Permission: authenticated user who created the order
func (s *service) GetOrder(ctx context.Context, req *order.GetOrderRequest) (*order.GetOrderResponse, error) {
    // Authorizer user
    userID, err := s.authorize(ctx)
    if err != nil {
        return nil, err
    }
    // Get order from database
    orderID := req.GetId()
    if orderID == "" {
        return nil, errors.New("order id is required")
    }
    resOrder, err := s.DBProvider.GetOrderById(orderID)
    if err != nil {
        return nil, err
    }
    // Check if user is authorized to get the order
    if resOrder.UserID != userID {
        return nil, errors.New("unauthorized")
    }
    return &order.GetOrderResponse{
        Order: resOrder.AsAPIOrder(),
    }, nil
}

Main Entry Point (`main.go`)

package main

import (
    "log"
    "net"
    "os"

    "github.com/joho/godotenv"
    "google.golang.org/grpc"
    "google.golang.org/grpc/credentials/insecure"

    order "github.com/lakhansamani/ecom-grpc-apis/order/v1"
    user "github.com/lakhansamani/ecom-grpc-apis/user/v1"

    "github.com/lakhansamani/ecom-grpc-orderd/db"
    "github.com/lakhansamani/ecom-grpc-orderd/service"
)

func main() {
    // Read .env file as environment variables
    err := godotenv.Load()
    if err != nil {
        log.Println(".env file not found, using environment variables")
    }

    // DB URL
    dbURL := os.Getenv("DB_URL")
    if dbURL == "" {
        log.Fatal("DB_URL is required")
    }
    // Initialize database
    dbProvider := db.New(dbURL)

    // Get User Service URL
    userServiceURL := os.Getenv("USER_SERVICE_URL")
    if userServiceURL == "" {
        log.Fatal("USER_SERVICE_URL is required")
    }

    // Create UserServiceClient using grpc
    grpcConn, err := grpc.NewClient(userServiceURL, grpc.WithTransportCredentials(
        insecure.NewCredentials(),
    ))
    if err != nil {
        log.Fatalf("Failed to dial UserService: %v", err)
    }
    defer grpcConn.Close()

    userServiceClient := user.NewUserServiceClient(grpcConn)

    // Create a new gRPC server
    server := grpc.NewServer()

    // Register OrderService with gRPC
    orderService := service.New(
        service.Config{},
        service.Dependencies{
            DBProvider:  dbProvider,
            UserService: userServiceClient,
        })
    order.RegisterOrderServiceServer(server, orderService)

    // Start gRPC server
    listener, err := net.Listen("tcp", ":50052")
    if err != nil {
        log.Fatalf("Failed to listen: %v", err)
    }
    log.Println("gRPC Server is running on port 50052...")
    if err := server.Serve(listener); err != nil {
        log.Fatalf("Failed to serve: %v", err)
    }
}

Environment Configuration (`.env`)

DB_URL=postgres://postgres:postgres@localhost:5432/orderdb
USER_SERVICE_URL=0.0.0.0:50051

Docker Setup

`.dockerignore`

/bin
/pkg

`Dockerfile`

FROM golang:1.23 AS builder
WORKDIR /app
COPY go.mod go.sum ./
RUN go mod download
COPY . .
RUN CGO_ENABLED=0 GOOS=linux go build -o orderd ./main.go

FROM alpine:latest
WORKDIR /app
RUN apk add --no-cache ca-certificates
COPY --from=builder /app/orderd .
EXPOSE 50052
CMD ["./orderd"]

Running the Service

go run main.go

docker build -t order-service .
docker run --env-file .env -p 50052:50052 order-service

Here are the commands to test it

grpcurl -plaintext -H "authorization: bearer JWT_TOKEN"  -d '{ "product": "book 1", "quantity": 1  }' -proto=apis/order/v1/order.proto localhost:50052 order.v1.OrderService/CreateOrder

grpcurl -plaintext -H "authorization: bearer JWT_TOKEN"  -d '{ "id": "ID"  }' -proto=apis/order/v1/order.proto localhost:50052 order.v1.OrderService/GetOrder

Code Link

Github APIs repo

Conclusion

Built an Order Service with authentication.
Used gRPC metadata for authorization.
Stored orders in PostgreSQL with GORM.

Next: We will learn Logging & Tracing!

Stay Tuned

Implementing User Service with go, gRPC and PostgreSQL - Part 3

Lakhan Samani — Wed, 05 Mar 2025 16:37:50 +0000

In Part 2 we introduced apis for User service, this post walks through implementing a User Service in Go using gRPC with PostgreSQL as the database. The service provides user authentication with JWT-based authentication. The implementation includes:

gRPC service definition
Database integration using GORM
JWT-based authentication
A structured service layer

Note: Please replace the github user name and repo in each of the following folder

Project Structure

The folder structure for the service is as follows:

ecom-grpc/userd/
│-- db/
│   │-- db.go
│   │-- user.go
│-- service/
│   │-- service.go
│   │-- login.go
│   │-- register.go
│   │-- me.go
│-- utils/
│   │-- jwt.go
│-- main.go
│-- .env
│-- Dockerfile
│-- .dockerignore

Database Provider (`db/db.go`)

This file defines the interface for database operations and initializes the connection to PostgreSQL using GORM.

package db

import (
    "log"

    "gorm.io/driver/postgres"
    "gorm.io/gorm"
)

// Provider defines the interface for the database provider
type Provider interface {
    CreateUser(user *User) (*User, error)
    GetUserByEmail(email string) (*User, error)
    GetUserByID(id string) (*User, error)
}

// provider implements the Provider interface
type provider struct {
    db *gorm.DB
}

// New creates new database provider
// connects to db and returns the provider
func New(dbURL string) Provider {
    db, err := gorm.Open(postgres.Open(dbURL), &gorm.Config{})
    if err != nil {
        log.Fatalf("Failed to connect to database: %v", err)
    }

    // Auto-migrate User model
    db.AutoMigrate(&User{})

    return &provider{db}
}

Database User Methods (`db/user.go`)

This file implements the methods to interact with the users table.

package db

import (
    "github.com/google/uuid"
    "golang.org/x/crypto/bcrypt"
    "gorm.io/gorm"

    user "github.com/lakhansamani/ecom-grpc-apis/user/v1"
)

// User represents the User model in DB
type User struct {
    ID       string `gorm:"primaryKey"`
    Name     string
    Email    string `gorm:"unique"`
    Password string
}

// AsAPIUser converts the User model to API User
func (u *User) AsAPIUser() *user.User {
    return &user.User{
        Id:    u.ID,
        Name:  u.Name,
        Email: u.Email,
    }
}

// BeforeSave GORM hook to hash password only if it's changed
func (u *User) BeforeSave(tx *gorm.DB) (err error) {
    // Hash the new password
    hashedPassword, err := bcrypt.GenerateFromPassword([]byte(u.Password), bcrypt.DefaultCost)
    if err != nil {
        return err
    }
    u.Password = string(hashedPassword)
    return nil
}

// Before create
func (u *User) BeforeCreate(tx *gorm.DB) (err error) {
    // Generate UUID
    u.ID = uuid.NewString()
    return
}

// CreateUser creates a new user in the database
func (p *provider) CreateUser(u *User) (*User, error) {
    err := p.db.Create(u).Error
    return u, err
}

// GetUserByEmail fetches a user by email from the database
func (p *provider) GetUserByEmail(email string) (*User, error) {
    var u User
    err := p.db.Where("email = ?", email).First(&u).Error
    return &u, err
}

// GetUserByID fetches a user by ID from the database
func (p *provider) GetUserByID(id string) (*User, error) {
    var u User
    err := p.db.Where("id = ?", id).First(&u).Error
    return &u, err
}

User Service (`service/service.go`)

This file defines the service layer for user operations. It holds configurations and dependencies.

package service

import (
    user "github.com/lakhansamani/ecom-grpc-apis/user/v1"
    "github.com/lakhansamani/ecom-grpc-userd/db"
)

type Config struct {
    JWTSecret string
}

type Dependencies struct {
    DBProvider db.Provider
}

// Service implements the User service.
type Service interface {
    user.UserServiceServer
}

type service struct {
    Config
    Dependencies
}

// New creates a new User service.
func New(cfg Config, deps Dependencies) Service {
    return &service{
        Config:       cfg,
        Dependencies: deps,
    }
}

Register API (`service/register.go`)

Handles user registration.

package service

import (
    "context"
    "errors"
    "strings"

    user "github.com/lakhansamani/ecom-grpc-apis/user/v1"

    "github.com/lakhansamani/ecom-grpc-userd/db"
)

// Register API to register a new user
// Permission: none
func (s *service) Register(ctx context.Context, req *user.RegisterRequest) (*user.RegisterResponse, error) {
    name := req.GetName()
    email := req.GetEmail()
    password := req.GetPassword()

    if strings.TrimSpace(name) == "" {
        return nil, errors.New("name is required")
    }

    if strings.TrimSpace(email) == "" {
        return nil, errors.New("email is required")
    }

    if strings.TrimSpace(password) == "" {
        return nil, errors.New("password is required")
    }

    resUser, err := s.DBProvider.CreateUser(&db.User{
        Name:     name,
        Email:    email,
        Password: password,
    })
    if err != nil {
        return nil, err
    }

    return &user.RegisterResponse{
        UserId: resUser.ID,
    }, nil
}

Login API (`service/login.go`)

Handles user login and JWT generation.

package service

import (
    "context"
    "errors"
    "strings"

    "golang.org/x/crypto/bcrypt"

    user "github.com/lakhansamani/ecom-grpc-apis/user/v1"

    "github.com/lakhansamani/ecom-grpc-userd/utils"
)

// Login API to login a user
// Permission: none
func (s *service) Login(ctx context.Context, req *user.LoginRequest) (*user.LoginResponse, error) {
    email := req.GetEmail()
    password := req.GetPassword()

    if strings.TrimSpace(email) == "" {
        return nil, errors.New("email is required")
    }

    if strings.TrimSpace(password) == "" {
        return nil, errors.New("password is required")
    }

    // Get user by email
    resUser, err := s.DBProvider.GetUserByEmail(email)
    if err != nil {
        return nil, err
    }

    // Match password
    if err := bcrypt.CompareHashAndPassword([]byte(resUser.Password), []byte(password)); err != nil {
        return nil, errors.New("invalid password")
    }

    // Generate JWT token
    token, err := utils.GenerateJWT(s.JWTSecret, resUser.ID)
    if err != nil {
        return nil, err
    }

    return &user.LoginResponse{
        Token: token,
    }, nil
}

Me API (`service/me.go`)

Retrieves the currently authenticated user.

package service

import (
    "context"
    "errors"
    "strings"

    user "github.com/lakhansamani/ecom-grpc-apis/user/v1"
    "github.com/lakhansamani/ecom-grpc-userd/utils"
    "google.golang.org/grpc/metadata"
)

// Me API to get user details
// Permission: authenticated user
func (s *service) Me(ctx context.Context, req *user.MeRequest) (*user.MeResponse, error) {
    // Get the Authorization bearer token from the context
    // Extract the token from the header
    md, ok := metadata.FromIncomingContext(ctx)
    if !ok {
        return nil, errors.New("missing metadata")
    }

    authHeader, exists := md["authorization"]
    if !exists || len(authHeader) == 0 {
        return nil, errors.New("missing authorization token")
    }

    token := authHeader[0]
    // Make sure the token is not empty and is bearer token
    if token == "" {
        return nil, errors.New("missing token")
    }
    tokenSplit := strings.Split(token, " ")
    if len(tokenSplit) != 2 {
        return nil, errors.New("invalid token")
    }
    if strings.ToLower(tokenSplit[0]) != "bearer" {
        return nil, errors.New("invalid token")
    }
    token = tokenSplit[1]
    userID, err := utils.VerifyJWT(s.JWTSecret, token)
    if err != nil {
        return nil, err
    }
    // Fetch the user from the database
    resUser, err := s.DBProvider.GetUserByID(userID)
    if err != nil {
        return nil, err
    }
    // Return the user details
    return &user.MeResponse{
        User: resUser.AsAPIUser(),
    }, nil
}

JWT Utility (`utils/jwt.go`)

Handles JWT generation and verification.

package utils

import (
    "time"

    "github.com/golang-jwt/jwt"
)

// Generate JWT token
func GenerateJWT(secret, userID string) (string, error) {
    token := jwt.NewWithClaims(jwt.SigningMethodHS256, jwt.MapClaims{
        "user_id": userID,
        "exp":     time.Now().Add(time.Hour * 24).Unix(),
    })
    return token.SignedString([]byte(secret))
}

// VerifyJWT verifies the JWT token
func VerifyJWT(secret, tokenString string) (string, error) {
    token, err := jwt.Parse(tokenString, func(token *jwt.Token) (interface{}, error) {
        return []byte(secret), nil
    })
    if err != nil {
        return "", err
    }
    claims, ok := token.Claims.(jwt.MapClaims)
    if !ok {
        return "", err
    }
    // Check if token is valid
    if !token.Valid {
        return "", err
    }
    return claims["user_id"].(string), nil
}

Main File (`main.go`)

Starts the gRPC server and initializes dependencies.

package main

import (
    "log"
    "net"
    "os"

    "github.com/joho/godotenv"
    "google.golang.org/grpc"

    userpb "github.com/lakhansamani/ecom-grpc-apis/user/v1"

    "github.com/lakhansamani/ecom-grpc-userd/db"
    "github.com/lakhansamani/ecom-grpc-userd/service"
)

func main() {
    // Read .env file as environment variables
    err := godotenv.Load()
    if err != nil {
        log.Println(".env file not found, using environment variables")
    }

    // DB URL
    dbURL := os.Getenv("DB_URL")
    if dbURL == "" {
        log.Fatal("DB_URL is required")
    }
    // JWT Secret
    jwtSecret := os.Getenv("JWT_SECRET")
    if jwtSecret == "" {
        log.Fatal("JWT_SECRET is required")
    }
    // Initialize database
    dbProvider := db.New(dbURL)

    // Create a new gRPC server
    server := grpc.NewServer()

    // Register UserService with gRPC
    userService := service.New(
        service.Config{
            JWTSecret: jwtSecret,
        },
        service.Dependencies{
            DBProvider: dbProvider,
        })
    userpb.RegisterUserServiceServer(server, userService)

    // Start gRPC server
    listener, err := net.Listen("tcp", ":50051")
    if err != nil {
        log.Fatalf("Failed to listen: %v", err)
    }
    log.Println("gRPC Server is running on port 50051...")
    if err := server.Serve(listener); err != nil {
        log.Fatalf("Failed to serve: %v", err)
    }
}

Dockerfile (Dockerfile)

This file helps in creating userd container image that we can use in future with kubernetes.

# Build Stage
FROM golang:1.23 AS builder

WORKDIR /app

# Copy go.mod and go.sum and download dependencies
COPY go.mod go.sum ./
RUN go mod download

# Copy source code
COPY . .

# Build the Go binary with static linking (Alpine compatible)
RUN CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -o userd ./main.go

# Final Runtime Stage (Alpine)
FROM alpine:latest

WORKDIR /app

# Install certificates (required for HTTPS calls)
RUN apk add --no-cache ca-certificates

# Copy binary from builder
COPY --from=builder /app/userd .

# Expose gRPC port
EXPOSE 50051

# Run the application
CMD ["./userd"]

Docker ignore file (.dockerignore)

This file helps ignoring development files example .env in production build

.env

Here is .env file for local development

DB_URL=postgres://postgres:postgres@localhost:5432/userdb
JWT_SECRET=secret

Running the Service

docker run --name postgres-cluster -e POSTGRES_PASSWORD=postgres -p 5432:5432 -d postgres
docker exec -it postgres-cluster psql -U postgres -c "CREATE DATABASE userdb;"
go run main.go

Now, your User Service is live with gRPC, PostgreSQL, and JWT authentication! 🚀

You can try following commands with correct .proto file path

grpcurl -plaintext -d '{ "name": "John Doe", "email": "john@example.com", "password": "securepass" }' -proto=apis/user/v1/user.proto localhost:50051 user.v1.UserService/Register

grpcurl -plaintext -d '{ "email": "john@example.com", "password": "securepass" }' -proto=apis/user/v1/user.proto localhost:50051 user.v1.UserService/Login

grpcurl -plaintext -H "authorization: bearer JWT_TOKEN" -proto=apis/user/v1/user.proto localhost:50051 user.v1.UserService/Me

Code Link

Github APIs repo

🎯 Next Steps

🚀 Now that we have the User Service implemented, in Part 4, we will:
✅ Implement Order Service.

Defining gRPC APIs with Protobuf - Part 2

Lakhan Samani — Tue, 04 Mar 2025 11:43:39 +0000

In Part 1, we set up our project structure. Now, we’ll:
✅ Define User and Order APIs
✅ Generate gRPC code from .proto files.

We have created apis folder so lets navigate to that folder and define our apis there.

cd ecom-grpc/apis

Note: Please replace the github user name and repo in each of the following folder

Step 1: Install Protobuf Compiler

Before writing .proto files, install protoc:

🔹 Mac (Homebrew)

brew install protobuf

🔹 Ubuntu/Linux

sudo apt update && sudo apt install -y protobuf-compiler

🔹 Windows

Download from protobuf releases.

Verify installation:

protoc --version

Step 2: Define User Service API

create user/v1/user.proto file in apis repo

mkdir -p user/v1/
touch user/v1/user.proto

syntax = "proto3";

package user.v1;

option go_package = "github.com/lakhansamani/ecom-grpc-apis/user/v1";

// User Service definition
service UserService {
    // Register API to register a new user
    // Permission: none
    rpc Register(RegisterRequest) returns (RegisterResponse);
    // Login API to login a user
    // Permission: none
    rpc Login(LoginRequest) returns (LoginResponse);
    // Me API to get user details
    // Permission: authenticated user
    rpc Me(MeRequest) returns (MeResponse);
}

// User model
message User {
    string id = 1;
    string name = 2;
    string email = 3;
}

// Register API
message RegisterRequest {
    string name = 1;
    string email = 2;
    string password = 3;
}

message RegisterResponse {
    string user_id = 1;
}

// Login API
message LoginRequest {
    string email = 1;
    string password = 2;
}

message LoginResponse {
    string token = 1;
}

// Me API (token will be taken from gRPC metadata)
message MeRequest {}

message MeResponse {
    User user = 1;
}

New API methods

Register → Registers a new user and returns a JWT token.
Login → Authenticates a user and returns a JWT token.
Me → Returns user details using the JWT token.

🔹 Step 3: Define Order Service API

create order/v1/order.proto file in apis repo

mkdir -p order/v1/
touch order/v1/order.proto

syntax = "proto3";

package order.v1;

option go_package = "github.com/lakhansamani/ecom-grpc-apis/order/v1";

service OrderService {
    // CreateOrder API to create a new order
    // Permission: authenticated user
    rpc CreateOrder(CreateOrderRequest) returns (CreateOrderResponse);
    // GetOrder API to get order details
    // Permission: authenticated user who created the order
    rpc GetOrder(GetOrderRequest) returns (GetOrderResponse);
}

// Order model
message Order {
    string id = 1;
    string user_id = 2;
    string product = 3;
    int32 quantity = 4;
    double unit_price = 5;
}

// CreateOrder request API
message CreateOrderRequest {
    string product = 1;
    int32 quantity = 2;
}

// CreateOrder response API
message CreateOrderResponse {
    Order order = 1;
}

// GetOrder request API
message GetOrderRequest {
    string id = 1;
}

// GetOrder response API
message GetOrderResponse {
    Order order = 1;
}

New API methods

CreateOrder - to create order
GetOrder - to get order information

🔹 Step 4: Generate gRPC Code

To simplify the process lets create a Makefile at the root of apis repo which we can run to generate the go-grpc code.

touch Makefile

all: build
build:
    protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=require_unimplemented_servers=false:. --go-grpc_opt=paths=source_relative user/v1/user.proto
    protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=require_unimplemented_servers=false:. --go-grpc_opt=paths=source_relative order/v1/order.proto

Run make command to generate the files and push to github.

Code Link

Github APIs repo

🎯 Next Steps

🚀 Now that we have the User & Order Service protobuf, in Part 3, we will:
✅ Implement User Service.
✅ Implement Order Service.

Stay tuned! 🚀

Building a Distributed E-commerce App with Go and gRPC – Part 1

Lakhan Samani — Mon, 03 Mar 2025 13:13:56 +0000

Introduction & Project Overview

🚀 Why Build a Distributed System with Go & gRPC?

Modern applications need to be scalable, fast, and efficient. A monolithic approach may work initially, but as your app grows, it becomes hard to maintain. This is where microservices come in!

In this series, we’ll build a simple distributed e-commerce system using Go, gRPC, PostgreSQL, and Kubernetes. By the end of this series, you’ll understand how to:

✅ Design gRPC APIs for communication between services.
✅ Implement services with Go and handle user authentication.
✅ Use middleware for logging, monitoring, and security.
✅ Deploy services to Kubernetes and scale them dynamically.

📌 Project Overview: What Are We Building?

We’ll build a simplified e-commerce backend with two core services:

User Service 🧑‍💻
- Create users
- Authenticate users with JWT
- Retrieve user details

Order Service 📦
- Place an order (linked to a user)
- Get order details

🛠 Tech Stack

Technology	Purpose
Go	Backend services
gRPC	Communication between services
PostgreSQL	Database for users & orders
Redis	Caching & rate-limiting
Prometheus & Grafana	Monitoring & metrics
Prometheus & Alertmanager	Setting up alerts for failures
Kubernetes (K8s)	Deployment & auto-scaling

🌍 System Architecture

Here’s how the services interact:

The User Service stores user data in PostgreSQL.
The Order Service processes orders and links them to users.
Both services communicate via gRPC instead of REST for high performance.

📝 Step 1: Setting Up the Project

1️⃣ Install Dependencies

Make sure you have Go installed (v1.23+ recommended). Then install proto buff:

go install google.golang.org/protobuf/cmd/protoc-gen-go@latest
go install google.golang.org/grpc/cmd/protoc-gen-go-grpc@latest

Now add them to your PATH:

export PATH="$PATH:$(go env GOPATH)/bin"

2️⃣ Create the Project Structure

Run the following to set up the project:

Note: please replace github.com/lakhansamani with your github or go package URL.

mkdir ecom-grpc
cd ecom-grpc
mkdir apis userd orderd
cd apis # common repo for all the service apis 
go mod init github.com/lakhansamani/ecom-grpc-apis
cd ..
cd userd # user service
go mod init github.com/lakhansamani/ecom-grpc-userd
cd ..
cd orderd # order service
go mod init github.com/lakhansamani/ecom-grpc-orderd

🎯 Next Steps
In Part 2, we’ll define our gRPC APIs using Protobuf and implement the User Service in Go. Stay tuned! 🚀

Why Microservices Are Important Compared to Monolithic Architecture

Lakhan Samani — Sun, 25 Aug 2024 05:47:40 +0000

In today's fast-paced technological landscape, businesses need scalable and flexible solutions to adapt quickly to changing demands. This is where microservices architecture shines compared to the traditional monolithic approach.

1. What is Monolithic Architecture?

Monolithic architecture is a single, unified system where all components are interconnected and interdependent. This means that any changes or updates to the system require the entire application to be rebuilt and redeployed. While this approach may work for smaller projects, it can become a bottleneck as the system grows.

2. What is Microservices Architecture?

Microservices architecture, on the other hand, breaks down a large application into smaller, independent services that can be developed, deployed, and scaled independently. Each service focuses on a specific business functionality and communicates with other services through APIs. This modular approach offers several advantages:

3. Benefits of Microservices over Monolithic Architecture

Scalability: Microservices allow you to scale specific parts of your application independently, rather than scaling the entire system. This targeted scaling can lead to more efficient resource usage and cost savings.
Flexibility and Agility: With microservices, different teams can work on different services simultaneously, using different technologies if needed. This accelerates development and allows for faster iteration and deployment.
Improved Fault Isolation: In a microservices architecture, if one service fails, it doesn't necessarily bring down the entire application. This isolation makes the system more resilient and easier to maintain.
Technology Diversity: Microservices enable the use of different programming languages, frameworks, or databases for different services, making it easier to adopt new technologies.

How Golang Helps in Microservices

Golang (Go) has become a popular choice for developing microservices due to its simplicity, performance, and concurrency model. Here's how Golang fits well with the microservices architecture:

Performance: Go's lightweight nature and efficient execution model make it ideal for building high-performance microservices. Its compiled binaries are fast and have a small memory footprint.
Concurrency: Go's goroutines and channels provide a simple yet powerful way to handle concurrent tasks, which is a common requirement in microservices for handling multiple requests simultaneously.
Scalability: Go's efficient resource management and quick start-up time make it easier to scale services. Its built-in tools and libraries simplify the process of scaling and deploying microservices.
Simplicity and Maintainability: Go's straightforward syntax and minimalistic design make it easier to read, understand, and maintain code, which is crucial in a microservices environment where multiple teams may be working on different services.

Common Mistakes in Microservices

While microservices offer many benefits, there are also pitfalls to watch out for:

Over-Engineering: One of the common mistakes is creating too many microservices from the start. It's essential to find the right balance and avoid unnecessary complexity.
Poor Service Boundaries: Defining clear boundaries for each service is critical. Overlapping responsibilities or tightly coupled services can lead to the same problems as monolithic architecture.
Inefficient Communication: Microservices rely on network communication, which can introduce latency and increase the complexity of the system. It's essential to optimize communication and handle network failures gracefully.
Lack of Monitoring and Logging: With many services running independently, monitoring and logging are crucial for tracking performance and diagnosing issues. Without proper observability, it becomes challenging to maintain the system.
Ignoring Data Management Challenges: Managing data consistency across services can be challenging. It's important to design with eventual consistency in mind and use appropriate strategies for data synchronization.

Conclusion

Microservices architecture offers a scalable, flexible, and resilient approach to building modern applications. Golang's performance, concurrency model, and simplicity make it a great choice for developing microservices. However, careful planning and execution are required to avoid common pitfalls and fully leverage the benefits of microservices. By understanding these challenges and making informed decisions, organizations can successfully transition from monolithic to microservices architecture.

Authorize NodeJS APIs using Authorizer

Lakhan Samani — Tue, 23 May 2023 08:35:04 +0000

In this blog post, we will learn how to authorize users for API calls, based on valid sessions and roles using Authorizer. Authorizer is an open-source database-independent auth solution. You can bring your database and have authentication and authorization service ready out of the box.

For the demo purpose, we will be using [Express](https://expressjs.com/) server and writing a middleware that will validate user sessions & roles based on the JWT (JSON Web Token) in the Authorization header.

Refer to the source code on github.

Step 1: Setup Authorizer Instance

Deploy production-ready Authorizer instance using one-click deployment options available below

Infra provider	One-click link	Additional information
Railway.app		docs
Heroku		docs
Render		docs

For more information & deployment options like docker / Kubernetes / helm charts refer to the docs

Configure Instance

Open the Authorizer instance endpoint in a browser
SignUp as an Admin with a secure password
Configure environment variables from the dashboard. Check env docs for more information.

Note: DATABASE_URL , DATABASE_TYPE , REDIS_URL are the variables that can only be configured as the authorizer instance's system environment variable.

Step 2: Create Express App

Note: This step is optional if you already have a Nodejs application up and running

Setup project

# Create directory
mkdir my-apis

# Change directory
cd my-apis

# Initialize nodejs APP
npm init -y

# Install express
npm install express

# Create index.js
touch index.js

Add Start Command to package.json

{
  "name": "my-apis",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "node index.js",
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "Lakhan Samani",
  "license": "ISC",
  "dependencies": {
    "express": "^4.18.2"
  }
}

Setup Express App and Create a basic API in index.js

// index.js
const express = require('express');

const app = express();
const port = `3000`;

app.get('/', (req, res) => {
  res.send('Hello World');
});

app.listen(port, () => {
  console.log(`[server]: Server is running at http://localhost:${port}`);
});

Step 3: Create Authorization Middleware

Install @authorizerdev/authorizer-js

npm i --save @authorizerdev/authorizer-js

Create auth_middleware.js
```
touch auth_middleware.js
```

Implement authorization middleware

// auth_middleware.js

const { Authorizer } = require("@authorizerdev/authorizer-js");

const authRef = new Authorizer({  
  authorizerURL: "AUTHORIZER_URL_FROM_STEP 1",
  redirectURL: "FRONTEND_URL",
  clientID: "AUTHORIZER_CLIENT_ID FROM DASHBOARD"
});

const authMiddleware = async (req, res, next) => {
    const authHeader = req.headers.authorization;
  if (!authHeader) {
    return res.status(403).json({ error: "Authorization not found" });
  }

  const splitHeader = authHeader.split(" ");
  if (splitHeader.length != 2) {
    return res.status(403).json({ error: "Invalid auth header" });
  }

  if (splitHeader[0].toLowerCase() != "bearer") {
    return res.status(403).json({ error: "Bearer token not found" });
  }

  const token = splitHeader[1];
  // Validate jwt token via authorizer sdk
  try {
    const res = await authRef.validateJWTToken({
      token,
      token_type: "id_token", // This can be access_token, refresh_token
       // roles: [user] // specify roles that you want to validate jwt for, by default it will just verify jwt.
    });
    req.user = res.claims;
  } catch (err) {
    console.error(err);
    return res.status(403).json({ error: "Invalid JWT token" });
  }

  next();
}

module.exports = authMiddleware

Step 4: Add auth middleware to APIs

Update index.js with following content

// index.js
const express = require('express');
const authMiddleware = require('./auth_middleware')

const app = express();
const port = `3000`;

app.get('/', authMiddleware, (req, res) => {
  res.send('Hello World');
});

app.listen(port, () => {
  console.log(`[server]: Server is running at http://localhost:${port}`);
});

Step 5: Test the API

Start API Server
```
npm start
```
Make a curl request
```
curl http://localhost:3000
```

Note: This will return an error, as we have not specified the Authorization header with a valid JWT token

For this test to pass we need to have a valid JWT token, so let's generate a valid JWT by making a login call to the authorizer server

# Replace Authorizer URL from step 1
# Replace credentials with right credentials in --data-raw (demo@yopmail.com, Test@123#)
# If you have no user on your instance, first signup using AUTHORIZER_URL_FROM_STEP_1/app?redirect_uri=AUTHORIZER_URL_FROM_STEP_1/app

curl --location --request POST 'AUTHORIZER_URL_FROM_STEP_1/graphql' \
--header 'Content-Type: application/json' \
--data-raw '{"query":"mutation login {\n  login(params: {\n    email: \"demo@yopmail.com\",\n    password: \"Test@123#\"\n  }) {\n    id_token\n  }\n}","variables":{}}'

This should return an id_token in response, as shown in the screenshot below. Copy the id_token from the response and pass it to our nodeJS API server

Test the API

curl --header 'Authorization: Bearer TOKEN_COPIED_FROM_ABOVE_STEP' http://localhost:3000

That's all I hope this helps you authorize your APIs easily and make them secure based on the correct session and roles.

For more information check out the links below.

Website: https://authorizer.dev
Docs: https://docs.authorizer.dev
Github: https://github.com/authorizerdev/authorizer
React SDK: https://github.com/authorizerdev/authorizer-react
Javascript SDK: https://github.com/authorizerdev/authorizer-js
Youtube: https://youtube.com/playlist?list=PLSQGbUjHc6bpaAgCiQPzNxiUPr7SkDAFR
Examples: https://github.com/authorizerdev/examples/
Discord: https://discord.gg/Zv2D5h6kkK
Github Sponsorship: https://github.com/sponsors/authorizerdev/
Buy me Coffee: https://www.buymeacoffee.com/lakhansamani

DEV Community: Lakhan Samani

Your AI assistant can leak files people aren't allowed to see — here's how to fix it

The stack — open source end to end

Permissions are relationships, not roles

How does it work? The architecture

Fail closed, or don't bother

Revocation that actually revokes

What we verified (and what we'd tell you anyway)

FAQ

Do I need to re-index Qdrant when permissions change?

What happens if Authorizer is unreachable mid-request?

Can a prompt-injected agent bypass the filter?

Does this work with LangChain or LlamaIndex?

What are the current limits?

Run the demo yourself, in four steps

Links

Your AI assistant can leak files people aren't allowed to see — here's how to fix it

The stack — open source end to end

Permissions are relationships, not roles

How does it work? The architecture

Fail closed, or don't bother

Revocation that actually revokes

What we verified (and what we'd tell you anyway)

FAQ

Do I need to re-index Qdrant when permissions change?

What happens if Authorizer is unreachable mid-request?

Can a prompt-injected agent bypass the filter?

Does this work with LangChain or LlamaIndex?

What are the current limits?

Run the demo yourself, in four steps

Links

Authorizer v2 Is Here: Self-Hosted Auth, Rebuilt From the Ground Up

Why We Built v2

What's in v2

13+ Database Backends

11 Social Login Providers

Multi-Factor Authentication

Full OAuth 2.0 / OIDC Compliance

Developer Experience

Role-Based Access Control

One-Command Deployment

What Changed From v1

What's Coming Next: The Roadmap

Phase 1: Security Hardening

Phase 2: Authorization & Machine-to-Machine

Phase 3: Enterprise SSO & Federation

Phase 4: AI-Era Authentication

Phase 5: Advanced Security & Modern Standards (Q2-Q3 2027)

Why Self-Hosted Auth Matters in 2026

Get Started

Join the Community

Sponsor Authorizer

Deploying Our Distributed System on Kubernetes with Kind - Part 7

Why Deploy on Kubernetes?

Setting Up the Kind Cluster

Step 1: Install Kind & kubectl

Step 2: Create a Kind Cluster

Deploying Databases (PostgreSQL)

Deploying the User and Order Services

userd Deployment & Service userd.yaml

orderd Deployment & Service orderd.yaml

Deploying Observability Stack

Jaeger (Tracing) jaeger.yaml

Prometheus (Metrics) prometheus.yaml

Alertmanager (Alerts) alertmanager.yaml

Accessing Services

Repository

Conclusion

Implementing monitoring and alerting for distributed system - Part 6

Introduction

Why Prometheus?

What kind of metrics are we interested in?

Integrating Metrics in userd

Step 1: Configure Prometheus Go Client

Step 2: Define custom metrics for API service for userd service

Step 3: Call metrics in service handlers

Integrating Metrics in orderd

Step 1 remains same as userd service.

Step 2: Add gRPC Client Interceptors

Step 3: Define metrics for orderd services

userd Deployment & Service `userd.yaml`

orderd Deployment & Service `orderd.yaml`

Jaeger (Tracing) `jaeger.yaml`

Prometheus (Metrics) `prometheus.yaml`

Alertmanager (Alerts) `alertmanager.yaml`

Step 2: Define custom metrics for API service for `userd` service

Step 1 remains same as `userd` service.

Step 3: Define metrics for `orderd` services

Step 1: Create `alert_rules.yml`

Step 2: Create `alertmanager.yml`

Step 3: Update `prometheus.yml`

Step 4: Start `alertmanager` & Restart `prometheus`

Database Layer (`db/`)

`db/db.go` - Database Connection

`db/order.go` - Order Model

Service Layer (`service/`)

`service/service.go` - Service Dependencies

`service/create_order.go` - Create Order API

`service/get_order.go` - Get Order API

Main Entry Point (`main.go`)

Environment Configuration (`.env`)

`.dockerignore`

`Dockerfile`

Database Provider (`db/db.go`)

Database User Methods (`db/user.go`)

User Service (`service/service.go`)

Register API (`service/register.go`)

Login API (`service/login.go`)

Me API (`service/me.go`)

JWT Utility (`utils/jwt.go`)

Main File (`main.go`)