DEV Community: Aniefon Umanah

Making AI Classify Jobs Before Scoring Them: A Two-Model Architecture

Aniefon Umanah — Sat, 28 Feb 2026 09:19:15 +0000

The problem with auto-applying to jobs is that a generic resume gets ignored. The insight: different roles need different emphasis. A startup wants to see velocity and ownership. An enterprise team wants to see process and scale. Same resume, different angles.

I built a job archetype classifier that runs before scoring. It detects what type of engineer the company actually wants, then tells the scoring system which parts of your background to emphasize.

The Seven Archetypes

Not all "Senior Engineer" roles are the same. I defined seven hiring intents:

type JobArchetype =
  | 'STARTUP_BUILDER'           // Move fast, ship features, wear hats
  | 'ENTERPRISE_ENGINEER'        // Scale, reliability, process
  | 'SAAS_PLATFORM_ENGINEER'     // Multi-tenant, API design, growth
  | 'FINTECH_TRANSACTION_ENGINEER' // Compliance, accuracy, security
  | 'FRONTEND_PRODUCT_ENGINEER'  // UX craft, A/B tests, metrics
  | 'BACKEND_INFRA_ENGINEER'     // Uptime, throughput, observability
  | 'FULLSTACK_GENERALIST'       // Breadth over depth

Each archetype maps to what the company is really hiring to mitigate. A fintech doesn't want your "move fast and break things" story. A seed-stage startup doesn't care about your SOC2 implementation.

The Classification Flow

Before scoring, every job goes through classification:

interface ArchetypeResult {
  primaryArchetype: JobArchetype;
  secondaryArchetype: JobArchetype | null;
  confidence: number;
  resumeWeighting: ResumeWeighting;
}

interface ResumeWeighting {
  emphasize: string[];      // "distributed systems", "team leadership"
  deEmphasize: string[];    // "rapid prototyping", "hackathons"
  toneBias: string;         // "reliability_and_correctness"
}

The LLM reads the job description and returns both the archetype AND specific instructions for how to angle your resume. These weightings get stored on the job:

@Column({ type: 'varchar', length: 50, nullable: true })
archetype: JobArchetype | null;

@Column({ type: 'jsonb', nullable: true })
resumeWeighting: ResumeWeighting | null;

Why Two Models?

I'm using different LLMs for classification vs scoring:

// Main LLM for scoring (needs reasoning depth)
this.client = new OpenAI({
  apiKey: this.configService.get('llm.apiKey'),
  baseURL: this.configService.get('llm.baseUrl'),
});

// Cheaper classifier (pattern matching, less reasoning)
this.classifierClient = new OpenAI({
  apiKey: this.configService.get('llm.classifier.apiKey'),
  baseURL: this.configService.get('llm.classifier.baseUrl'),
});

Classification is cheaper and faster than scoring. I can run it on every job without blowing the budget. The classifier just needs to recognize patterns in job descriptions. The scorer needs to reason about fit.

The System Prompt Strategy

The classifier uses a scoring system, not vibes:

1. Read the job posting thoroughly.
2. Identify hiring intent signals: keywords, phrases, company context.
3. Score each archetype:
   - Trigger keyword match = +2 points
   - Context signal match = +3 points  
   - Negative signal = -2 points
4. Select primary archetype (highest score)
5. Select secondary if within 30% of primary

Each archetype has specific signals. For STARTUP_BUILDER:

Triggers: "founding", "0-to-1", "early stage", "move fast"
Contexts: Mentions small team size, equity compensation, ambiguity
Negatives: "Fortune 500", "established processes", "legacy"

The model counts signal matches and outputs a confidence score. Low confidence (< 50) means the posting is too generic or vague.

What Actually Changed in the Code

The big addition is the ResearchModule that handles pre-scoring analysis. When a job moves from QUEUED to RESEARCHING:

async classifyArchetype(
  jobText: string,
  companyContext?: string
): Promise<ArchetypeResult> {
  const response = await this.classifierClient.chat.completions.create({
    model: this.classifierModel,
    messages: [
      { role: 'system', content: ARCHETYPE_SYSTEM_PROMPT },
      { role: 'user', content: this.buildClassificationPrompt(jobText, companyContext) }
    ],
    tools: [CLASSIFY_ARCHETYPE_TOOL],
    tool_choice: { type: 'function', function: { name: 'save_archetype_classification' }}
  });

  // Parse tool call response...
}

The job entity stores the classification result:

// In Job entity
markResearching(): void {
  if (this.status !== ApplicationStatus.QUEUED) {
    throw new InvalidJobStatusError(/*...*/);
  }
  this.status = ApplicationStatus.RESEARCHING;
}

markResearchComplete(archetype: JobArchetype, weighting: ResumeWeighting): void {
  this.archetype = archetype;
  this.resumeWeighting = weighting;
  this.status = ApplicationStatus.APPLYING;
}

The POC That Validated This

I wrote archetype-eval.poc.ts to test three models on classification accuracy:

const MODELS = [
  { name: 'GLM-4.7-Flash', inputCost: 0.07, outputCost: 0.40 },
  { name: 'GLM-5', inputCost: 1.0, outputCost: 3.2 },
  { name: 'Kimi K2.5', inputCost: 0.6, outputCost: 3.0 }
];

GLM-4.7-Flash won: 94% accuracy at 7x lower cost. The classification task doesn't need the reasoning depth of the larger models.

Why This Matters

Generic applications get filtered out. Humans scan for relevance in 10 seconds. The archetype classification gives the system the context it needs to highlight the right 10 seconds of your background.

A job at a payments company gets classified as FINTECH_TRANSACTION_ENGINEER. The scoring system then knows to emphasize your "implemented idempotent payment processing" experience over your "shipped features fast" startup work.

Same resume. Different emphasis. Better match rate.

The code's live at github.com/Anietex/huntly.

Feature Gating: How We Built a Freemium SaaS Without Duplicating Components

Aniefon Umanah — Sat, 13 Dec 2025 01:27:52 +0000

The problem with most feature-gating implementations? They end up scattering billing checks throughout your codebase like landmines. You've got if (user.plan === 'pro') everywhere, making it impossible to test components in isolation or understand what features are actually gated.

Here's how we solved it with a FeatureGate component that wraps restricted features instead of modifying them.

The Problem

We needed to add subscription tiers to our analytics dashboard. Some features should show upgrade prompts on free plans, others should be completely hidden. But we didn't want to:

Touch every component that needs restrictions
Break existing tests
Make components aware of billing logic
Duplicate UI for "locked" states

The Solution: A Wrapper Component

Instead of modifying components to check access, we wrap them:

// Before: No restrictions
<DeviceBreakdown />

// After: Gated by feature flag
<FeatureGate feature="device_analytics">
  <DeviceBreakdown />
</FeatureGate>

The component itself stays pure. The billing logic lives in one place.

Two Modes of Gating

We built two behaviors into FeatureGate:

Mode 1: Hide completely (default)

<FeatureGate feature="ai_analysis" mode="hide">
  <AIPlayground />
</FeatureGate>

If the user doesn't have access, the component doesn't render. The UI flows naturally without gaps.

Mode 2: Show upgrade prompt

<FeatureGate feature="ab_tests" mode="replace">
  <ABTestBuilder />
</FeatureGate>

This shows a standardized upgrade card with the plan requirement and CTA.

The Hook That Powers It

const { hasAccess, isLoading, planName } = useFeatureAccess('reading_insights');

This hook:

Checks the current user's plan
Returns whether they have access to the feature
Provides loading states
Gives us the plan name for messaging

Handling Page-Level Gates

For full pages that require upgrades, we added checks at the route level:

export default function ComparePage() {
  const { hasAccess, isLoading, planName } = useFeatureAccess('document_comparison');

  if (!isLoading && !hasAccess) {
    return (
      <div className="upgrade-prompt">
        <Lock className="h-8 w-8" />
        <h3>Document Comparison</h3>
        <p>Currently on: {planName}</p>
        <Button onClick={() => router.push('/settings/subscription')}>
          Upgrade to Business
        </Button>
      </div>
    );
  }

  // Normal page content...
}

Early return pattern keeps the locked state isolated at the top.

What Changed in This Commit

Looking at the analytics page specifically:

// Before
<DeviceBreakdown />
<LocationBreakdown />
<BrowserBreakdown />
<VisitorTable />

// After
<FeatureGate feature="device_analytics">
  <DeviceBreakdown />
</FeatureGate>
<FeatureGate feature="location_analytics">
  <LocationBreakdown />
</FeatureGate>
<FeatureGate feature="browser_analytics">
  <BrowserBreakdown />
</FeatureGate>
<FeatureGate feature="reading_insights">
  <VisitorTable />
</FeatureGate>

Each analytics widget is independently gated. Free users see basic metrics, paid users see the full breakdown.

Testing Benefits

The biggest win? Our components stay testable:

// Component test - no billing logic
it('renders device breakdown', () => {
  render(<DeviceBreakdown />);
  expect(screen.getByText('Mobile')).toBeInTheDocument();
});

// Integration test - with feature gate
it('hides device breakdown for free users', () => {
  mockUser({ plan: 'free' });
  render(
    <FeatureGate feature="device_analytics">
      <DeviceBreakdown />
    </FeatureGate>
  );
  expect(screen.queryByText('Mobile')).not.toBeInTheDocument();
});

The Gotcha: Early Returns and Hooks

We hit an issue in the link detail page. Initial code:

const { hasAccess } = useFeatureAccess('reading_insights');

// 🚫 This violates Rules of Hooks
if (!hasAccess) {
  return <UpgradePrompt />;
}

const someOtherHook = useSomeHook(); // Hook called conditionally!

The fix: Call all hooks first, then check access:

const { hasAccess } = useFeatureAccess('reading_insights');
const someOtherHook = useSomeHook();
const router = useRouter();

// ✅ Now we can return early safely
if (!hasAccess) {
  return <UpgradePrompt />;
}

Configuration Lives in One Place

All feature definitions live in a single config:

const FEATURE_ACCESS = {
  free: ['basic_analytics'],
  pro: ['basic_analytics', 'reading_insights', 'device_analytics'],
  business: ['basic_analytics', 'reading_insights', 'device_analytics', 'document_comparison', 'ab_tests'],
};

Want to change what's included in Pro? Update one object.

The Result

17 files modified with feature gates
Zero changes to the actual feature components
Consistent upgrade prompts across the app
Clean separation between features and billing

The wrapper pattern keeps billing concerns isolated. Components stay focused on what they do, not who can see them.

Building Reliable Stripe Subscriptions in NestJS: Webhook Idempotency and Optimistic Locking

Aniefon Umanah — Fri, 28 Nov 2025 21:35:04 +0000

Subscriptions power modern SaaS apps, but integrating payment providers like Stripe requires handling webhooks reliably. In a recent commit to the CommitLore backend, I implemented a full subscription system using Stripe. The standout technical choice was ensuring idempotent webhook processing to avoid duplicate updates during retries or failures.

This approach uses a dedicated webhook_events table to track Stripe events and optimistic locking on the subscriptions table to prevent race conditions. It's a practical pattern for any NestJS app dealing with external async events. Let's break it down.

Why Idempotency Matters for Stripe Webhooks

Stripe webhooks notify your server of events like subscription upgrades, cancellations, or payment failures. But networks are unreliable—events can arrive multiple times, out of order, or during downtime. Without safeguards, this leads to duplicate charges, inconsistent subscription states, or infinite loops.

The solution: Store each webhook in a database first, process it only once, and update the subscription atomically. This commit adds:

A new webhook_events table to log incoming events with status tracking.
Optimistic locking via a version column on subscriptions to detect concurrent modifications.

This prevents issues like double-processing a customer.subscription.updated event, which could incorrectly downgrade a plan.

The Database Schema: Tracking Webhook Events

The migration creates a simple but robust table. Here's the key SQL from the TypeORM migration:

// src/database/migrations/1764287633270-AddWebhookEventsAndOptimisticLocking.ts
await queryRunner.query(`
  CREATE TABLE "webhook_events" (
    "id" uuid NOT NULL DEFAULT uuid_generate_v4(),
    "createdAt" TIMESTAMP NOT NULL DEFAULT now(),
    "updatedAt" TIMESTAMP NOT NULL DEFAULT now(),
    "stripeEventId" character varying(255) NOT NULL,
    "eventType" character varying(100) NOT NULL,
    "status" character varying(20) NOT NULL DEFAULT 'processing',
    "errorMessage" text,
    "processedAt" TIMESTAMP,
    CONSTRAINT "UQ_2321876280160661b78693399a5" UNIQUE ("stripeEventId"),
    CONSTRAINT "PK_4cba37e6a0acb5e1fc49c34ebfd" PRIMARY KEY ("id")
  )
`);
await queryRunner.query(`CREATE INDEX "IDX_2321876280160661b78693399a" ON "webhook_events" ("stripeEventId")`);
await queryRunner.query(`ALTER TABLE "subscriptions" ADD "version" integer NOT NULL DEFAULT 1`);

Unique stripeEventId: Ensures no duplicates—Stripe guarantees this ID is unique per event.
Status enum (processing, processed, failed): Tracks lifecycle for retries or debugging.
Index on stripeEventId: Fast lookups during webhook receipt.

The entity looks like this:

// src/modules/subscriptions/domain/entities/webhook-event.entity.ts
@Entity('webhook_events')
export class WebhookEvent extends BaseEntity {
  @Column({ type: 'varchar', length: 255, unique: true })
  @Index()
  stripeEventId: string;

  @Column({ type: 'varchar', length: 100 })
  eventType: string;

  @Column({ type: 'varchar', length: 20, default: WebhookEventStatus.PROCESSING })
  status: WebhookEventStatus;

  @Column({ type: 'text', nullable: true })
  errorMessage?: string;

  @Column({ type: 'timestamp', nullable: true })
  processedAt?: Date;
}

Handling Webhooks in the Use Case

The handle-stripe-webhook.use-case.ts (expanded in this commit) follows a clear flow:

Verify and Parse: Use Stripe's SDK to construct the event from payload and signature.
Idempotency Check: Query webhook_events by stripeEventId. If already processed, return 200 early.
Store Event: Insert as "processing".
Process Event: Switch on eventType to update subscriptions (e.g., upgrade on customer.subscription.updated).
Atomic Update: Use optimistic locking—TypeORM's @VersionColumn() throws if the version mismatches.
Mark Complete: Update to "processed" or "failed" with error details.

Here's a simplified excerpt of the processing logic:

// Pseudo-code from handle-stripe-webhook.use-case.ts
async execute(payload: Buffer, signature: string) {
  const event = stripe.constructWebhookEvent(payload, signature);
  const existing = await this.webhookEventRepository.findByStripeEventId(event.id);

  if (existing?.status === WebhookEventStatus.PROCESSED) {
    return { success: true }; // Idempotent skip
  }

  const webhookEvent = await this.webhookEventRepository.create({
    stripeEventId: event.id,
    eventType: event.type,
  });

  try {
    switch (event.type) {
      case 'customer.subscription.updated':
        await this.updateSubscription(subscriptionId, event.data.object);
        break;
      // ... other cases like 'invoice.payment_failed'
    }
    await this.webhookEventRepository.updateStatus(webhookEvent.id, WebhookEventStatus.PROCESSED, undefined, new Date());
  } catch (error) {
    await this.webhookEventRepository.updateStatus(webhookEvent.id, WebhookEventStatus.FAILED, error.message);
    throw error;
  }
}

The optimistic lock shines in updateSubscription:

// In subscription entity
@VersionColumn()
version: number; // Increments on each save, fails save if concurrent change detected

If two webhooks hit simultaneously (rare but possible during retries), TypeORM rolls back the conflicting update, letting the other succeed.

Cleanup and Monitoring

To avoid table bloat, a CleanupWebhookEventsUseCase deletes processed events older than 30 days:

// src/modules/subscriptions/application/use-cases/cleanup-webhook-events.use-case.ts
async execute(retentionDays: number = 30): Promise<number> {
  const cutoffDate = new Date();
  cutoffDate.setDate(cutoffDate.getDate() - retentionDays);
  return await this.webhookEventRepository.deleteProcessedOlderThan(cutoffDate);
}

Schedule this via cron in production. For monitoring, log failures and expose metrics on unprocessed events.

Gotchas and Lessons

Signature Verification: Always verify webhook signatures first—Stripe's test mode uses different keys.
Retry Logic: Stripe retries failed webhooks, so idempotency is non-negotiable. But don't process "failed" events again unless you implement custom retries.
Version Conflicts: In high-traffic apps, optimistic locking might conflict often; consider pessimistic if needed, but it's overkill here.
Testing: Use Stripe CLI for local webhooks. Mock the repo in unit tests to simulate duplicates.

This setup scales well—events are lightweight, and the unique constraint catches issues early. For a NestJS + Stripe combo, it's a solid foundation. If you're building payments, start with this pattern to save debugging headaches later.

Adding WordPress to a Multi-Platform Publishing System

Aniefon Umanah — Thu, 27 Nov 2025 19:51:39 +0000

When you're building a content publishing API that already supports Twitter, LinkedIn, and Dev.to, adding WordPress feels like it should be straightforward. It's just another REST API, right?

Not quite.

The WordPress REST API Quirk

Unlike the other platforms, WordPress doesn't use OAuth. It uses HTTP Basic Authentication with application passwords. This means instead of managing tokens and refresh flows, you're dealing with a simpler but more permanent credential system.

Here's what the authentication looks like:

const authString = Buffer.from(`${username}:${applicationPassword}`).toString('base64');

const response = await fetch(`${siteUrl}/wp-json/wp/v2/posts`, {
  method: 'POST',
  headers: {
    'Authorization': `Basic ${authString}`,
    'Content-Type': 'application/json',
  },
  body: JSON.stringify(postData),
});

No token exchange, no scopes, no refresh logic. Just base64 encode the credentials and send them. This is both simpler and riskier—if those credentials leak, someone has write access to your WordPress site until you revoke them.

The Markdown Problem

The other platforms in this system accept plain text or markdown. WordPress expects HTML. This created a choice: require HTML input for WordPress posts, or convert markdown on the fly.

I went with conversion:

private markdownToHtml(markdown: string): string {
  let html = markdown;

  // Headers
  html = html.replace(/^### (.+)$/gm, '<h3>$1</h3>');
  html = html.replace(/^## (.+)$/gm, '<h2>$1</h2>');
  html = html.replace(/^# (.+)$/gm, ''); // Title handled separately

  // Code blocks
  html = html.replace(/```
{% endraw %}
(\w+)?\n([\s\S]*?)
{% raw %}
```/g, (_, lang, code) => {
    return `<pre><code class="language-${lang || 'plaintext'}">${code.trim()}</code></pre>`;
  });

  // Inline code
  html = html.replace(/`([^`]+)`/g, '<code>$1</code>');

  // Bold and italic
  html = html.replace(/\*\*(.+?)\*\*/g, '<strong>$1</strong>');
  html = html.replace(/\*(.+?)\*/g, '<em>$1</em>');

  // Paragraphs
  html = html.split('\n\n').map(p => `<p>${p.trim()}</p>`).join('\n');

  return html;
}

This is basic—it doesn't handle nested lists, tables, or complex markdown features. For a production system, you'd use a proper parser like marked. But it covers the common case: headers, code blocks, inline code, and basic formatting.

Error Handling That Actually Helps

When WordPress authentication fails, the REST API returns a 401. But there are multiple reasons why: wrong password, wrong username, application passwords not enabled, or even the REST API being disabled entirely.

The error handling differentiates these:

if (response.status === 401) {
  return { 
    success: false, 
    error: 'WordPress authentication failed. Check your username and application password.' 
  };
}
if (response.status === 403) {
  return { 
    success: false, 
    error: 'WordPress access forbidden. Ensure the user has permission to create posts.' 
  };
}
if (response.status === 404) {
  return { 
    success: false, 
    error: 'WordPress REST API not found. Ensure REST API is enabled and URL is correct.' 
  };
}

The 404 case is particularly important. Some WordPress sites disable the REST API for security reasons. Knowing that upfront saves debugging time.

The Title Extraction

WordPress posts require a separate title field. Other platforms either don't have titles (Twitter) or extract them from the content (Dev.to). This meant extracting the title from markdown:

private extractTitle(content: string, providedTitle?: string): string {
  if (providedTitle?.trim()) {
    return providedTitle.trim();
  }

  // Look for # Title format
  const lines = content.split('\n');
  if (lines.length > 0 && lines[0].startsWith('# ')) {
    return lines[0].substring(2).trim();
  }

  // Fallback: use first 60 chars
  const firstLine = lines[0]?.trim() || '';
  return firstLine.length > 60 
    ? firstLine.substring(0, 57) + '...' 
    : firstLine || 'Untitled Post';
}

If the content starts with # Title, use that. Otherwise, take the first line. If that's too long, truncate it. Always have a fallback.

What This Reveals About Platform Abstraction

The IPublisher interface treats all platforms the same:

interface IPublisher {
  supports(platform: Platform): boolean;
  publish(request: PublishRequest): Promise<PublishResult>;
}

But each implementation is wildly different. Twitter has character limits and no formatting. LinkedIn wants structured posts with hashtags. Dev.to needs front matter and tags. WordPress wants HTML and separate titles.

The abstraction holds because it's minimal. It doesn't try to hide platform differences—it just provides a common shape for the publishing flow. The complexity lives in each publisher service where it belongs.

Adding WordPress meant writing a new service that handles its specific quirks while conforming to the same interface. No changes to the core publishing logic. Just register the new publisher:

this.publishers = [
  twitterPublisher, 
  devtoPublisher, 
  linkedinPublisher, 
  wordpressPublisher
];

That's the benefit of designing around contracts instead of specifics.

Publishing to LinkedIn Without the Official API: A Cookie-Based Approach

Aniefon Umanah — Thu, 27 Nov 2025 18:25:29 +0000

The LinkedIn API for posting is notoriously restricted. You need to apply for partnership access, and even then, you're limited to organization pages. For individual developer accounts trying to automate their own posts? You're out of luck.

So I built a workaround: using browser cookies and Playwright to automate publishing to LinkedIn as if you're using the actual website.

The Core Problem

LinkedIn's official API won't let you post to personal profiles. But every developer knows that where there's a browser session, there's a way. The challenge is capturing that session in a way that's both secure and reliable.

The Cookie Capture Strategy

Instead of storing passwords or dealing with OAuth flows that don't grant the permissions we need, I'm capturing the active browser session:

interface LinkedInExtensionConnectInput {
  userId: string;
  cookies: LinkedInCookie[];
  userAgent: string;
}

The key cookies we need are li_at (LinkedIn authentication token) and JSESSIONID. These are captured from an active browser session via a Chrome extension, then stored for automation:

const liAtCookie = cookies.find((c) => c.name === 'li_at');
const jSessionCookie = cookies.find((c) => c.name === 'JSESSIONID');

if (!liAtCookie || !jSessionCookie) {
  throw new BadRequestException(
    'Missing required LinkedIn session cookies.'
  );
}

The Publishing Flow

Here's where it gets interesting. With Playwright, we can replay that browser session:

const browser = await chromium.launch({ headless: false });

const context = await browser.newContext({
  userAgent: credentials.userAgent
});

// Inject the captured cookies
const playwrightCookies = this.convertCookies(credentials.cookies);
await context.addCookies(playwrightCookies);

Now we have a browser that's "logged in" as the user, without ever touching their password.

The LinkedIn UI Challenge

LinkedIn's UI changes frequently, which makes selector-based automation fragile. The solution? Try multiple selectors until one works:

const startPostSelectors = [
  '[placeholder="Start a post"]',
  'div[data-placeholder="Start a post"]',
  '.share-box-feed-entry__top-bar',
  '.share-creation-state__text-field',
  'div[role="button"]:has-text("Start a post")',
  'button:has-text("Start a post")',
];

let startPostButton = null;
for (const selector of startPostSelectors) {
  try {
    startPostButton = await page.waitForSelector(selector, { 
      timeout: 2000 
    });
    if (startPostButton && await startPostButton.isVisible()) {
      break;
    }
  } catch {
    continue;
  }
}

This defensive approach means when LinkedIn tweaks their HTML, we don't immediately break.

Handling Session Expiry

The li_at cookie has an expiration date. We extract and store it:

const expiresAt = liAtCookie.expires
  ? new Date(liAtCookie.expires * 1000).toISOString()
  : undefined;

Then before publishing, we check:

if (credentials.expiresAt && new Date(credentials.expiresAt) < new Date()) {
  return {
    success: false,
    error: 'LinkedIn session has expired. Please reconnect.'
  };
}

This prevents failed publish attempts and tells the user exactly what they need to do.

The Tradeoff

This approach sacrifices the elegance of a proper API integration for something that actually works. It's more brittle—LinkedIn could change their UI tomorrow. It requires a headless browser, which means more resources. And users need to reconnect via the extension when their session expires.

But it works. For developers who want to automate their personal LinkedIn presence, that's what matters.

The alternative is manually posting or not posting at all. Given that choice, a little browser automation doesn't seem so bad.

Encrypting Secrets in Production (Without Breaking Everything)

Aniefon Umanah — Thu, 27 Nov 2025 12:05:43 +0000

I just spent way more time than I'd like to admit adding encryption to a NestJS app that was already live. The kind of feature that sounds simple until you're staring at a database full of plaintext API keys wondering how to migrate them without taking the whole thing offline.

Here's what actually happened.

The "Oh Shit" Moment

We store webhook secrets for GitHub and API credentials for Twitter/LinkedIn/Dev.to. All sitting in a PostgreSQL jsonb column. Unencrypted.

Not ideal? Sure. Security vulnerability? Absolutely. Something anyone actually exploits in a small B2B SaaS? Probably not, but still.

The real trigger was adding more integrations. Each new platform meant more credentials, more API keys, more things that could leak. At some point you have to stop pretending you'll "add encryption later."

The Part Nobody Tells You About

Here's the thing about encrypting existing data: you can't just flip a switch. You have to handle:

Existing plaintext data - Can't just encrypt in place, you'll break production
Backward compatibility - App needs to read both formats during migration
Column size - Encrypted data is bigger (way bigger)
Zero downtime - Can't just take the DB offline on a Tuesday

Every tutorial shows you how to encrypt new data. Nobody shows you how to migrate the old stuff while keeping the lights on.

The Solution (That Actually Worked)

Built a custom TypeORM transformer that auto-detects whether data is encrypted:

export class EncryptedStringTransformer implements ValueTransformer {
  to(value: string | null): string {
    if (!value) return '';
    // Check if already encrypted (has our format markers)
    if (this.isEncrypted(value)) return value;

    // Encrypt plaintext
    const iv = randomBytes(12);
    const cipher = createCipheriv('aes-256-gcm', key, iv);
    const encrypted = Buffer.concat([
      cipher.update(value, 'utf8'),
      cipher.final(),
    ]);
    const authTag = cipher.getAuthTag();

    // Format: iv:authTag:ciphertext (all base64)
    return `${iv.toString('base64')}:${authTag.toString('base64')}:${encrypted.toString('base64')}`;
  }

  from(value: string | null): string {
    if (!value) return '';
    // If not encrypted format, return as-is (backward compat)
    if (!this.isEncrypted(value)) return value;

    // Decrypt
    const [ivB64, authTagB64, encryptedB64] = value.split(':');
    // ... decryption logic
  }
}

The magic: it reads both formats. Plaintext passes through untouched. Encrypted data gets decrypted. New writes always encrypt.

This meant I could:

Deploy the transformer
Run a script to UPDATE all records (triggers encryption)
Zero downtime, zero breaking changes

What Actually Broke

Of course it wasn't that smooth.

Issue #1: Column length

Original column: VARCHAR(255)
Encrypted webhook secret: ~200 chars

Seems fine, right? Wrong. The migration failed on a few records because some secrets were already near the limit. Encrypted versions didn't fit.

Had to bump to VARCHAR(500) first, then migrate data.

Issue #2: The jsonb trap

Some credentials were nested in jsonb:

{
  "accessToken": "abc123",
  "refreshToken": "xyz789"
}

You can't use TypeORM transformers on jsonb fields. Had to:

Add a new TEXT column for encrypted credentials
Write custom serialization
Drop the old jsonb column
Rename the new one

All while keeping the entity interface identical so the rest of the codebase didn't notice.

Issue #3: Environment variables

The encryption key comes from ENCRYPTION_KEY env var. Obvious, right?

Except in testing I kept getting decryption failures. Turns out the key was getting loaded after TypeORM initialized the transformers. Race condition.

Fix: Lazy-load the key in the transformer instead of at module init. Ugly but works.

The Things That Helped

Testing with real prod data - Exported sanitized records, tested migration locally first
Gradual rollout - Deployed transformer first, let it run for a day, then migrated data
Monitoring - Added logs for every decryption failure, caught edge cases fast

What I'd Do Differently

Start with encryption from day one? Sure, but that's not useful advice when you're already live.

Real lesson: Build backward compatibility into your transforms from the start. Even if you're not encrypting yet, make your transformers detect and handle legacy formats. Future you will thank you.

Also: Don't underestimate column size requirements. Encrypted data is ~2-3x larger depending on encoding. Budget for it.

The Bonus Problem: Activity Logging

While I was in there, I added an activity feed. Every action (draft created, account connected, credentials updated) gets logged with full context.

Why mention this? Because it nearly broke the encryption migration.

The activity logger runs in a DB transaction. If it fails, the whole operation rolls back. Which meant every credentials update that should have triggered encryption... didn't, because the activity log couldn't serialize the metadata.

Had to make activity logging async and fire-and-forget. If it fails, whatever, we lose an activity log entry. Better than blocking critical operations.

The Result

~9,000 lines of changes. Most of it:

Migration files (careful, methodical schema changes)
Test fixtures (encryption breaks deterministic tests)
Activity logging infrastructure (the thing I didn't plan for)

Actual encryption code? Maybe 200 lines.

The rest is ceremony around not breaking production.

The real takeaway: Retrofitting security into a live system isn't about the encryption algorithm. It's about the 10 other things that break when you change how data is stored. The column sizes, the backward compatibility, the test fixtures, the race conditions.

Budget 10x the time you think it'll take. Then add logging so you can debug the things you didn't anticipate.

Tags: #nestjs #security #encryption #database #migration